Skip to content

Commit

Permalink
Prepare the original PR of @gbroccolo for the merge
Browse files Browse the repository at this point in the history
Move brin.c(h) files into src subdirectory

Add brin support in upgrade script

Fix some code problems

Fix test_init

Fix Indices section in the doc

Add test for sbox BRIN index

Fix found problems in the doc after review

Fix BRIN support for sbox
  • Loading branch information
Vitaly Davydov committed Aug 26, 2023
1 parent 6ebf5c2 commit fa429a5
Show file tree
Hide file tree
Showing 11 changed files with 337 additions and 73 deletions.
8 changes: 4 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ MODULE_big = pg_sphere
OBJS = src/sscan.o src/sparse.o src/sbuffer.o src/vector3d.o src/point.o \
src/euler.o src/circle.o src/line.o src/ellipse.o src/polygon.o \
src/path.o src/box.o src/output.o src/gq_cache.o src/gist.o \
src/key.o src/gnomo.o src/epochprop.o brin.o
src/key.o src/gnomo.o src/epochprop.o src/brin.o

ifneq ($(USE_HEALPIX),0)
OBJS += src/healpix.o src/moc.o src/process_moc.o \
Expand All @@ -33,7 +33,7 @@ DATA_built = $(RELEASE_SQL) \
DOCS = README.pg_sphere COPYRIGHT.pg_sphere
REGRESS = init tables points euler circle line ellipse poly path box index \
contains_ops contains_ops_compat bounding_box_gist gnomo epochprop \
contains overlaps spoint_brin
contains overlaps spoint_brin sbox_brin

ifneq ($(USE_HEALPIX),0)
REGRESS += healpix moc mocautocast
Expand All @@ -43,7 +43,7 @@ REGRESS_9_5 = index_9.5 # experimental for spoint3

TESTS = init_test tables points euler circle line ellipse poly path box \
index contains_ops contains_ops_compat bounding_box_gist gnomo \
epochprop contains overlaps
epochprop contains overlaps spoint_brin sbox_brin

ifneq ($(USE_HEALPIX),0)
TESTS += healpix moc mocautocast
Expand Down Expand Up @@ -262,7 +262,7 @@ endif
pg_sphere--1.2.2--1.2.3.sql:
cat upgrade_scripts/$@.in > $@

pg_sphere--1.2.3--1.3.0.sql:
pg_sphere--1.2.3--1.3.0.sql: pgs_brin.sql.in
cat upgrade_scripts/$@.in > $@

# end of local stuff
Expand Down
4 changes: 2 additions & 2 deletions README.pg_sphere
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ It provides:
* Object rotation by Euler angles
* Indexing of spherical data types

This is an R-Tree implementation using GiST for spherical objects like
This is an R-tree implementation using GiST for spherical objects like
spherical points and spherical circles with useful functions and operators.
It also support the Block Range INdexing (BRIN) for large datasets.
It also supports the Block Range INdexing (BRIN) for large datasets.

NOTICE:
This version will work only with PostgreSQL version 10 and above.
Expand Down
30 changes: 14 additions & 16 deletions doc/indices.sgm
Original file line number Diff line number Diff line change
Expand Up @@ -10,17 +10,17 @@
<application>pgSphere</application> uses <literal>GiST</literal>
and Block Range INdexing (<literal>BRIN</literal>) algorithms to create
spherical indices.
<literal>GiST</literal> index represents the R-Tree implementation for
spherical objects, while <literal>BRINs</literal> are based on "summarization"
<literal>GiST</literal> indexes utilize an R-tree implementation for
spherical objects, while <literal>BRIN</literal> indexes are based on the "summarization"
of data blocks (<literal>pages</literal>) on physical storage in order to
organize data searches on ranges of summarized data that can be easily skipped
on the base of search filters (see <ulink
url="https://www.postgresql.org/docs/9.5/static/brin-intro.html"><citetitle>
PostgreSQL documentation</citetitle></ulink> for further details on BRINs).
As a consequence, BRINs result to be really small indexes (up to 1000 times
than GiST ones), generally with lower a performance compared with a GiST one,
url="https://www.postgresql.org/docs/current/brin-intro.html"><citetitle>
PostgreSQL documentation</citetitle></ulink> for further details on BRIN indexes).
As a consequence, BRIN indexes are very small indexes (up to 1000 times smaller
than GiST ones), generally with lower performance compared with a GiST one,
but up to 100 times faster than a full sequential scan of a table performed
without any index. So BRINs are particularly suitable in a big data context.
without any index. So BRIN indexes are particularly suitable in a big data context.
An index speeds up the execution time of searches based on operators <link
linkend="op.over"><literal>&lt;@</literal></link>, <link
linkend="op.over"><literal>@</literal></link>, <link
Expand All @@ -29,9 +29,6 @@
linkend="op.equal"><literal>=</literal></link>, and <link
linkend="op.equal"><literal>!=</literal></link>.
</para>
<title>
Create a spherical index
</title>
<para>
You can create a GiST index with the following spherical data types:
</para>
Expand Down Expand Up @@ -87,17 +84,18 @@
<![CDATA[VACUUM ANALYZE test;]]>
</programlisting>
<para>
BRINs can be created through the following syntax:
BRIN index can be created through the following syntax:
</para>
<programlisting>
<![CDATA[CREATE INDEX test_pos_idx USING BRIN ON test (pos);]]>
</programlisting>
<para>
By default, BRINs summarize block of 128 pages. The lower numbers
of pages are specified, the higher granularity is reached during
the searches, and performance's gap between GiST indexes and BRINs
is lower (consider that BRINs size increases as well). Different
summarizations can be used with the following command:
By default, BRIN indexes summarize blocks of 128 pages. The smaller the
number of pages specified, the higher the granularity in searches,
and the gap in performance between GiST indexes and BRIN indexes will be
decreased. Note that the size of the BRIN indexes increases as well.
Different summarizations can be specified with the following
command:
</para>
<programlisting>
<![CDATA[CREATE INDEX test_pos_idx USING BRIN ON test (pos) WITH (pages_per_range = 16);]]>
Expand Down
4 changes: 2 additions & 2 deletions expected/init_test_healpix.out.in
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
psql:pg_sphere.test.sql:9271: NOTICE: return type smoc is only a shell
psql:pg_sphere.test.sql:9277: NOTICE: argument type smoc is only a shell
psql:pg_sphere.test.sql:9684: NOTICE: return type smoc is only a shell
psql:pg_sphere.test.sql:9690: NOTICE: argument type smoc is only a shell
91 changes: 91 additions & 0 deletions expected/sbox_brin.out
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
SELECT set_sphere_output_precision(8);
set_sphere_output_precision
-----------------------------
SET 8
(1 row)

CREATE TABLE test_boxes (
b sbox
);
COPY test_boxes (b) FROM stdin;
CREATE OR REPLACE FUNCTION qnodes(q text) RETURNS text
LANGUAGE 'plpgsql' AS
$$
DECLARE
exp TEXT;
mat TEXT[];
ret TEXT[];
BEGIN
FOR exp IN EXECUTE 'EXPLAIN ' || q
LOOP
--RAISE NOTICE 'EXP: %', exp;
mat := regexp_matches(exp, ' *(?:-> *)?(.*Scan on (test_boxes|test_boxes_brin_idx))');
--RAISE NOTICE 'MAT: %', mat;
IF mat IS NOT NULL THEN
ret := array_append(ret, mat[1]);
END IF;
--RAISE NOTICE 'RET: %', ret;
END LOOP;
RETURN array_to_string(ret,',');
END;
$$;
CREATE INDEX test_boxes_brin_idx ON test_boxes USING brin (b);
SET enable_indexscan = OFF;
SET enable_bitmapscan = OFF;
SET enable_seqscan = ON;
SELECT 'scan_seq', qnodes('SELECT * FROM test_boxes WHERE b <@ sbox ''( (10d,10d), (20d,20d) )''');
?column? | qnodes
----------+------------------------
scan_seq | Seq Scan on test_boxes
(1 row)

SELECT * FROM test_boxes WHERE b <@ sbox '( (10d,10d), (20d,20d) )';
b
---
(0 rows)

SELECT 'scan_seq', qnodes('SELECT * FROM test_boxes WHERE b && sbox ''( (10d,10d), (20d,20d) )''');
?column? | qnodes
----------+------------------------
scan_seq | Seq Scan on test_boxes
(1 row)

SELECT * FROM test_boxes WHERE b && sbox '( (10d,10d), (20d,20d) )';
b
--------------------------------------------------------
((0.34906585 , 0.17453293), (0.35006585 , 0.17463293))
(1 row)

SET enable_indexscan = OFF;
SET enable_bitmapscan = ON;
SET enable_seqscan = OFF;
SELECT 'scan_idx', qnodes('SELECT * FROM test_boxes WHERE b <@ sbox ''( (10d,10d), (20d,20d) )''');
?column? | qnodes
----------+-------------------------------------------------------------------------
scan_idx | Bitmap Heap Scan on test_boxes,Bitmap Index Scan on test_boxes_brin_idx
(1 row)

SELECT * FROM test_boxes WHERE b <@ sbox '( (10d,10d), (20d,20d) )';
b
---
(0 rows)

SELECT 'scan_idx', qnodes('SELECT * FROM test_boxes WHERE b && sbox ''( (10d,10d), (20d,20d) )''');
?column? | qnodes
----------+-------------------------------------------------------------------------
scan_idx | Bitmap Heap Scan on test_boxes,Bitmap Index Scan on test_boxes_brin_idx
(1 row)

SELECT * FROM test_boxes WHERE b && sbox '( (10d,10d), (20d,20d) )';
b
--------------------------------------------------------
((0.34906585 , 0.17453293), (0.35006585 , 0.17463293))
(1 row)

---- cleanup
DROP INDEX test_boxes_brin_idx;
DROP TABLE test_boxes;
DROP FUNCTION qnodes(text);
SET enable_indexscan = ON;
SET enable_bitmapscan = ON;
SET enable_seqscan = ON;
99 changes: 65 additions & 34 deletions expected/spoint_brin.out
Original file line number Diff line number Diff line change
@@ -1,56 +1,87 @@
CREATE TABLE
COPY 77
CREATE FUNCTION
CREATE INDEX
SET
SET
SET
?column? | qnodes
----------+----------
scan_seq | Seq Scan
(1 row)

p
CREATE TABLE test_points (
p spoint
);
COPY test_points (p) FROM stdin;
CREATE OR REPLACE FUNCTION qnodes(q text) RETURNS text
LANGUAGE 'plpgsql' AS
$$
DECLARE
exp TEXT;
mat TEXT[];
ret TEXT[];
BEGIN
FOR exp IN EXECUTE 'EXPLAIN ' || q
LOOP
--RAISE NOTICE 'EXP: %', exp;
mat := regexp_matches(exp, ' *(?:-> *)?(.*Scan on (test_points|brin_spoint))');
--RAISE NOTICE 'MAT: %', mat;
IF mat IS NOT NULL THEN
ret := array_append(ret, mat[1]);
END IF;
--RAISE NOTICE 'RET: %', ret;
END LOOP;
RETURN array_to_string(ret,',');
END;
$$;
CREATE INDEX brin_spoint ON test_points USING brin (p) WITH (pages_per_range = 16);
set enable_indexscan = off;
set enable_bitmapscan = off;
set enable_seqscan = on;
SELECT 'scan_seq', qnodes('SELECT * FROM test_points WHERE p <@ sbox ''( (10d,10d), (20d,20d) )''');
?column? | qnodes
----------+-------------------------
scan_seq | Seq Scan on test_points
(1 row)

SELECT * FROM test_points WHERE p <@ sbox '( (10d,10d), (20d,20d) )';
p
-----------------------------------------
(0.349065850398866 , 0.174532925199433)
(1 row)

?column? | qnodes
----------+----------
scan_seq | Seq Scan
SELECT 'scan_seq', qnodes('SELECT * FROM test_points WHERE p && sbox ''( (10d,10d), (20d,20d) )''');
?column? | qnodes
----------+-------------------------
scan_seq | Seq Scan on test_points
(1 row)

p
SELECT * FROM test_points WHERE p && sbox '( (10d,10d), (20d,20d) )';
p
-----------------------------------------
(0.349065850398866 , 0.174532925199433)
(1 row)

SET
SET
SET
?column? | qnodes
----------+------------------------------------
scan_idx | Bitmap Heap Scan,Bitmap Index Scan
set enable_indexscan = off;
set enable_bitmapscan = on;
set enable_seqscan = off;
SELECT 'scan_idx', qnodes('SELECT * FROM test_points WHERE p <@ sbox ''( (10d,10d), (20d,20d) )''');
?column? | qnodes
----------+------------------------------------------------------------------
scan_idx | Bitmap Heap Scan on test_points,Bitmap Index Scan on brin_spoint
(1 row)

p
SELECT * FROM test_points WHERE p <@ sbox '( (10d,10d), (20d,20d) )';
p
-----------------------------------------
(0.349065850398866 , 0.174532925199433)
(1 row)

?column? | qnodes
----------+------------------------------------
scan_idx | Bitmap Heap Scan,Bitmap Index Scan
SELECT 'scan_idx', qnodes('SELECT * FROM test_points WHERE p && sbox ''( (10d,10d), (20d,20d) )''');
?column? | qnodes
----------+------------------------------------------------------------------
scan_idx | Bitmap Heap Scan on test_points,Bitmap Index Scan on brin_spoint
(1 row)

p
SELECT * FROM test_points WHERE p && sbox '( (10d,10d), (20d,20d) )';
p
-----------------------------------------
(0.349065850398866 , 0.174532925199433)
(1 row)

DROP INDEX
DROP TABLE
DROP FUNCTION
SET
SET
SET
-- cleanup
DROP INDEX brin_spoint;
DROP TABLE test_points;
DROP FUNCTION qnodes(text);
set enable_indexscan = on;
set enable_bitmapscan = on;
set enable_seqscan = on;
6 changes: 4 additions & 2 deletions pgs_brin.sql.in
Original file line number Diff line number Diff line change
Expand Up @@ -406,6 +406,8 @@ ALTER OPERATOR FAMILY brin_inclusion_spheric_ops USING brin ADD
OPERATOR 7 @>(sbox, spoint),
OPERATOR 7 @>(spoint, sbox),
OPERATOR 8 <@(sbox, spoint),
OPERATOR 8 <@(spoint, sbox);

OPERATOR 8 <@(spoint, sbox),

OPERATOR 3 &&(sbox, sbox),
OPERATOR 7 @>(sbox, sbox),
OPERATOR 8 <@(sbox, sbox);
Loading

0 comments on commit fa429a5

Please sign in to comment.