一、軟件安裝
創(chuàng)新互聯(lián)是一家以網(wǎng)站建設(shè)公司、網(wǎng)頁設(shè)計、品牌設(shè)計、軟件運維、seo優(yōu)化、小程序App開發(fā)等移動開發(fā)為一體互聯(lián)網(wǎng)公司。已累計為水處理設(shè)備等眾行業(yè)中小客戶提供優(yōu)質(zhì)的互聯(lián)網(wǎng)建站和軟件開發(fā)服務(wù)。
1.軟件需求:
postgresql-9.5.2.tar.gz
pg_repack-1.3.4.zip
2.安裝pg_repack
[root@localhost pg_repack-1.3.4]# export PATH=/opt/pgsql/9.5.2/bin:$PATH
[root@localhost pg_repack-1.3.4]# export LD_LIBRARY_PATH=/opt/pgsql/9.5.2/lib
[root@localhost pg_repack-1.3.4]# export MANPATH=/opt/pgsql/9.5.2/share/man:$MANPATH
[root@localhost pg_repack-1.3.4]# make
make[1]: Entering directory `/home/soft/pg_repack-1.3.4/bin'
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -O2 -I/opt/pgsql/9.5.2/include -DREPACK_VERSION=1.3.4 -I. -I./ -I/opt/pgsql/9.5.2/include/server -I/opt/pgsql/9.5.2/include/internal -D_GNU_SOURCE -c -o pg_repack.o pg_repack.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -O2 -I/opt/pgsql/9.5.2/include -DREPACK_VERSION=1.3.4 -I. -I./ -I/opt/pgsql/9.5.2/include/server -I/opt/pgsql/9.5.2/include/internal -D_GNU_SOURCE -c -o pgut/pgut.o pgut/pgut.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -O2 -I/opt/pgsql/9.5.2/include -DREPACK_VERSION=1.3.4 -I. -I./ -I/opt/pgsql/9.5.2/include/server -I/opt/pgsql/9.5.2/include/internal -D_GNU_SOURCE -c -o pgut/pgut-fe.o pgut/pgut-fe.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -O2 pg_repack.o pgut/pgut.o pgut/pgut-fe.o -L/opt/pgsql/9.5.2/lib -lpq -L/opt/pgsql/9.5.2/lib -Wl,--as-needed -Wl,-rpath,'/opt/pgsql/9.5.2/lib',--enable-new-dtags -lpgcommon -lpgport -lz -lreadline -lrt -lcrypt -ldl -lm -o pg_repack
make[1]: Leaving directory `/home/soft/pg_repack-1.3.4/bin'
make[1]: Entering directory `/home/soft/pg_repack-1.3.4/lib'
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -O2 -fpic -DREPACK_VERSION=1.3.4 -I. -I./ -I/opt/pgsql/9.5.2/include/server -I/opt/pgsql/9.5.2/include/internal -D_GNU_SOURCE -c -o repack.o repack.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -O2 -fpic -DREPACK_VERSION=1.3.4 -I. -I./ -I/opt/pgsql/9.5.2/include/server -I/opt/pgsql/9.5.2/include/internal -D_GNU_SOURCE -c -o pgut/pgut-be.o pgut/pgut-be.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -O2 -fpic -DREPACK_VERSION=1.3.4 -I. -I./ -I/opt/pgsql/9.5.2/include/server -I/opt/pgsql/9.5.2/include/internal -D_GNU_SOURCE -c -o pgut/pgut-spi.o pgut/pgut-spi.c
( echo '{ global:'; gawk '/^[^#]/ {printf "%s;\n",$1}' exports.txt; echo ' local: *; };' ) >exports.list
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -O2 -fpic -shared -Wl,--version-script=exports.list -o pg_repack.so repack.o pgut/pgut-be.o pgut/pgut-spi.o -L/opt/pgsql/9.5.2/lib -Wl,--as-needed -Wl,-rpath,'/opt/pgsql/9.5.2/lib',--enable-new-dtags
sed 's,REPACK_VERSION,1.3.4,g' pg_repack.sql.in > pg_repack--1.3.4.sql;
sed 's,REPACK_VERSION,1.3.4,g' pg_repack.control.in > pg_repack.control
make[1]: Leaving directory `/home/soft/pg_repack-1.3.4/lib'
make[1]: Entering directory `/home/soft/pg_repack-1.3.4/regress'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/soft/pg_repack-1.3.4/regress'
[root@localhost pg_repack-1.3.4]# make install
make[1]: Entering directory `/home/soft/pg_repack-1.3.4/bin'
/bin/mkdir -p '/opt/pgsql/9.5.2/bin'
/usr/bin/install -c pg_repack '/opt/pgsql/9.5.2/bin'
make[1]: Leaving directory `/home/soft/pg_repack-1.3.4/bin'
make[1]: Entering directory `/home/soft/pg_repack-1.3.4/lib'
/bin/mkdir -p '/opt/pgsql/9.5.2/lib'
/bin/mkdir -p '/opt/pgsql/9.5.2/share/extension'
/bin/mkdir -p '/opt/pgsql/9.5.2/share/extension'
/usr/bin/install -c -m 755 pg_repack.so '/opt/pgsql/9.5.2/lib/pg_repack.so'
/usr/bin/install -c -m 644 .//pg_repack.control '/opt/pgsql/9.5.2/share/extension/'
/usr/bin/install -c -m 644 pg_repack--1.3.4.sql pg_repack.control '/opt/pgsql/9.5.2/share/extension/'
make[1]: Leaving directory `/home/soft/pg_repack-1.3.4/lib'
make[1]: Entering directory `/home/soft/pg_repack-1.3.4/regress'
make[1]: Nothing to be done for `install'.
make[1]: Leaving directory `/home/soft/pg_repack-1.3.4/regress'
[root@localhost pg_repack-1.3.4]#
3.創(chuàng)建初始環(huán)境
[postgres@localhost ~]$ createdb bloatdb
[postgres@localhost ~]$ psql -d bloatdb -c "create extension pgstattuple;"
CREATE EXTENSION
[postgres@localhost ~]$ psql -d bloatdb -c "CREATE EXTENSION pg_repack;"
CREATE EXTENSION
[postgres@localhost ~]$
$ psql bloatdb
psql (9.5.2)
Type "help" for help.
bloatdb=# \dx
List of installed extensions
Name | Version | Schema | Description
-------------+---------+------------+--------------------------------------------------------------
pg_repack | 1.3.4 | public | Reorganize tables in PostgreSQL databases with minimal locks
pgstattuple | 1.3 | public | show tuple-level statistics
plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language
(3 rows)
二、靜態(tài)(無活躍交易)膨脹整理測試
1.處理表tbl指定索引
1).準(zhǔn)備環(huán)境
bloatdb=# create table tbl(id int primary key, first varchar(20),second varchar(20));
CREATE TABLE
bloatdb=# create index idx_tbl_first on tbl (first);
CREATE INDEX
bloatdb=# create index idx_tbl_second on tbl (second);
CREATE INDEX
bloatdb=# SELECT count(*) FROM tbl;
count
-------
0
(1 row)
bloatdb=# SELECT pg_size_pretty(pg_total_relation_size('tbl'));
pg_size_pretty
----------------
24 kB
(1 row)
bloatdb=# INSERT INTO tbl VALUES(generate_series(1,10000), 'first'||(random()*(10^3))::integer, 'second'||(random()*(10^3))::integer);
INSERT 0 10000
bloatdb=# SELECT count(*) FROM tbl;
count
-------
10000
(1 row)
bloatdb=# SELECT pg_size_pretty(pg_total_relation_size('tbl'));
pg_size_pretty
----------------
1584 kB
(1 row)
bloatdb=#
更新列
bloatdb=# UPDATE tbl SET first= 'updated-001';
UPDATE 10000
bloatdb=# SELECT count(*) FROM tbl;
count
-------
10000
(1 row)
bloatdb=# SELECT pg_size_pretty(pg_total_relation_size('tbl'));
pg_size_pretty
----------------
3376 kB
(1 row)
bloatdb=#
2).查詢膨脹率
建立膨脹統(tǒng)計表
[postgres@localhost ~]$ /home/soft/pg_bloat_check-master/pg_bloat_check.py -c "dbname=bloatdb" --create_stats_table
膨脹統(tǒng)計
[postgres@localhost ~]$ /home/soft/pg_bloat_check-master/pg_bloat_check.py -c "dbname=bloatdb" -t tbl
1. public.idx_tbl_second.......................................................(52.69%) 417 kB wasted
2. public.idx_tbl_first........................................................(52.64%) 413 kB wasted
3. public.tbl_pkey.............................................................(57.79%) 388 kB wasted
[postgres@localhost ~]$
3).處理膨脹
指定數(shù)據(jù)庫的特定索引
[postgres@localhost ~]$ pg_repack -d bloatdb --index idx_tbl_first
INFO: repacking index "public"."idx_tbl_first"
[postgres@localhost ~]$ /home/soft/pg_bloat_check-master/pg_bloat_check.py -c "dbname=bloatdb" -t tbl
1. public.idx_tbl_second.......................................................(52.69%) 417 kB wasted
2. public.tbl_pkey.............................................................(57.79%) 388 kB wasted
3. public.idx_tbl_first.....................................................(0.93%) 3121 bytes wasted
[postgres@localhost ~]$
2.處理表tbl所有索引
1).準(zhǔn)備環(huán)境
bloatdb=# update tbl set second='chris';
UPDATE 10000
bloatdb=# SELECT count(*) FROM tbl;
count
-------
10000
(1 row)
bloatdb=# SELECT pg_size_pretty(pg_total_relation_size('tbl'));
pg_size_pretty
----------------
3600 kB
(1 row)
bloatdb=#
bloatdb=# update tbl set first='chris';
UPDATE 10000
bloatdb=# SELECT count(*) FROM tbl;
count
-------
10000
(1 row)
bloatdb=# SELECT pg_size_pretty(pg_total_relation_size('tbl'));
pg_size_pretty
----------------
4176 kB
(1 row)
bloatdb=#
2).檢查膨脹
[postgres@localhost ~]$ /home/soft/pg_bloat_check-master/pg_bloat_check.py -c "dbname=bloatdb" -t tbl
1. public.idx_tbl_second.......................................................(59.94%) 820 kB wasted
2. public.idx_tbl_first........................................................(40.94%) 409 kB wasted
3. public.tbl_pkey.............................................................(28.73%) 193 kB wasted
[postgres@localhost ~]$
3).處理tbl表所有索引膨脹
[postgres@localhost ~]$ pg_repack -d bloatdb --table tbl --only-indexes
INFO: repacking indexes of "tbl"
INFO: repacking index "public"."idx_tbl_first"
INFO: repacking index "public"."idx_tbl_second"
INFO: repacking index "public"."tbl_pkey"
[postgres@localhost ~]$ /home/soft/pg_bloat_check-master/pg_bloat_check.py -c "dbname=bloatdb" -t tbl
1. public.idx_tbl_first.....................................................(1.23%) 3028 bytes wasted
2. public.idx_tbl_second....................................................(1.23%) 3028 bytes wasted
3. public.tbl_pkey..........................................................(1.23%) 3028 bytes wasted
[postgres@localhost ~]$
3.處理tbl數(shù)據(jù)和索引膨脹
1).索引膨脹
[postgres@localhost ~]$ /home/soft/pg_bloat_check-master/pg_bloat_check.py -c "dbname=bloatdb" -t tbl
1. public.idx_tbl_first.........................................................(57.87%) 49 MB wasted
2. public.idx_tbl_second........................................................(39.29%) 34 MB wasted
3. public.tbl_pkey..............................................................(51.22%) 26 MB wasted
2).處理膨脹online VACUUM FULL 數(shù)據(jù)庫bloatdb表tbl(數(shù)據(jù)和索引)
[postgres@localhost ~]$ pg_repack --no-order --table tbl -d bloatdb
INFO: repacking table "tbl"
[postgres@localhost ~]$ /home/soft/pg_bloat_check-master/pg_bloat_check.py -c "dbname=bloatdb" -t tbl
1. public.tbl_pkey..............................................................(0.0%) 0 bytes wasted
2. public.idx_tbl_second........................................................(0.0%) 0 bytes wasted
3. public.idx_tbl_first.........................................................(0.0%) 0 bytes wasted
[postgres@localhost ~]$
三、動態(tài)(有交易發(fā)生時)膨脹處理
1.整個表做膨脹處理
1).初始條件
-- clear table data
bloatdb=# select * from tbl;
id | first | second
----+-------+--------
(0 rows)
bloatdb=#
bloatdb=# INSERT INTO tbl VALUES(generate_series(1,100000), 'first'||(random()*(10^3))::integer, 'second'||(random()*(10^3))::integer);
INSERT 0 100000
bloatdb=# UPDATE tbl SET first= 'updated-001';
UPDATE 100000
bloatdb=#
-- check bloat
[postgres@localhost ~]$ /home/soft/pg_bloat_check-master/pg_bloat_check.py -c "dbname=bloatdb" -t tbl
1. public.idx_tbl_second........................................................(67.26%) 17 MB wasted
2. public.idx_tbl_first.........................................................(67.46%) 17 MB wasted
3. public.tbl_pkey............................................................(63.91%) 9832 kB wasted
[postgres@localhost ~]$
2).大量插入數(shù)據(jù)同時做膨脹處理
statement_timeout=0, 視情況調(diào)整:maintenance_work_mem,wal_keep_segments(streaming,SSD<2000>)
先插入數(shù)據(jù),過程中處理膨脹加上-T參數(shù)值為3600.
-- session 1:insert data
bloatdb=# INSERT INTO tbl VALUES(generate_series(100001,3000000), 'first'||(random()*(10^3))::integer, 'second'||(random()*(10^3))::integer);
光標(biāo)閃爍
-- session 2:repack during insert
$ pg_repack -d bloatdb --no-order --table tbl --wait-timeout=3600
INFO: repacking table "tbl"
光標(biāo)閃爍
############################## args: -j ###########################################
如果使用--table指定多個table時,會依次處理每個指定的表。如果整理使用-j參數(shù),則pg_repack,在創(chuàng)建臨時表索引時會啟動多個后臺進程并行創(chuàng)建索引,一般每建立一個索引都需要啟動一個后臺進程,直到min(j,tbl_idx_number<表中總的索引數(shù)>)數(shù)量的worker被創(chuàng)建完成。當(dāng)指定j數(shù)量小于索引數(shù)量時,一個索引創(chuàng)建完成時,空閑的work會自動被分派去建立剩余索引。當(dāng)指定j數(shù)量大于索引數(shù)量時,一次性分派索引總數(shù)個work來執(zhí)行索引創(chuàng)建任務(wù)。
$ pg_repack -j 10 --no-order -d bloatdb --table tbl --wait-timeout=3600
NOTICE: Setting up workers.conns
INFO: repacking table "tbl"
LOG: Initial worker 0 to build index: CREATE UNIQUE INDEX index_22025 ON repack.table_22022 USING btree (id)
LOG: Initial worker 1 to build index: CREATE INDEX index_22027 ON repack.table_22022 USING btree (first)
LOG: Initial worker 2 to build index: CREATE INDEX index_22028 ON repack.table_22022 USING btree (second)
LOG: Command finished in worker 0: CREATE UNIQUE INDEX index_22025 ON repack.table_22022 USING btree (id)
LOG: Command finished in worker 1: CREATE INDEX index_22027 ON repack.table_22022 USING btree (first)
LOG: Command finished in worker 2: CREATE INDEX index_22028 ON repack.table_22022 USING btree (second)
$
指定多個表的情況,j < idx_numbers
$ pg_repack -j 2 --no-order -d bloatdb --table tbl -t tbl01 --wait-timeout=3600
NOTICE: Setting up workers.conns
INFO: repacking table "tbl"
LOG: Initial worker 0 to build index: CREATE UNIQUE INDEX index_22025 ON repack.table_22022 USING btree (id)
LOG: Initial worker 1 to build index: CREATE INDEX index_22027 ON repack.table_22022 USING btree (first)
LOG: Command finished in worker 0: CREATE UNIQUE INDEX index_22025 ON repack.table_22022 USING btree (id)
LOG: Assigning worker 0 to build index #2: CREATE INDEX index_22028 ON repack.table_22022 USING btree (second)
LOG: Command finished in worker 1: CREATE INDEX index_22027 ON repack.table_22022 USING btree (first)
LOG: Command finished in worker 0: CREATE INDEX index_22028 ON repack.table_22022 USING btree (second)
(處理過程中有長事務(wù),會等待事務(wù)完成)
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
NOTICE: Waiting for 1 transactions to finish. First PID: 10426
INFO: repacking table "tbl01"
LOG: Initial worker 0 to build index: CREATE UNIQUE INDEX index_22065 ON repack.table_22062 USING btree (id)
LOG: Initial worker 1 to build index: CREATE INDEX index_22067 ON repack.table_22062 USING btree (first)
LOG: Command finished in worker 0: CREATE UNIQUE INDEX index_22065 ON repack.table_22062 USING btree (id)
LOG: Assigning worker 0 to build index #2: CREATE INDEX index_22068 ON repack.table_22062 USING btree (second)
LOG: Command finished in worker 1: CREATE INDEX index_22067 ON repack.table_22062 USING btree (first)
LOG: Command finished in worker 0: CREATE INDEX index_22068 ON repack.table_22062 USING btree (second)
$
##################################################################################
--session 1 finish insert
bloatdb=# INSERT INTO tbl VALUES(generate_series(100001,3000000), 'first'||(random()*(10^3))::integer, 'second'||(random()*(10^3))::integer);
INSERT 0 2900000
bloatdb=#
-- session 2: finish repack
[postgres@localhost ~]$ pg_repack -d bloatdb --no-order --table tbl --wait-timeout=3600
INFO: repacking table "tbl"
-- session 2:膨脹檢查
[postgres@localhost ~]$ /home/soft/pg_bloat_check-master/pg_bloat_check.py -c "dbname=bloatdb" -t tbl
1. public.tbl_pkey..............................................................(0.0%) 0 bytes wasted
2. public.idx_tbl_second........................................................(0.0%) 0 bytes wasted
3. public.idx_tbl_first.........................................................(0.0%) 0 bytes wasted
[postgres@localhost ~]$
-- session 1: 數(shù)據(jù)檢查
bloatdb=# select count(*) from tbl ;
count
---------
3000000
(1 row)
bloatdb=#
2.指定tbl表所有索引膨脹處理
如果tbl表有多個索引情況下,默認(rèn)處理方式,一個索引接著一個索引做膨脹處理即使指定了-j參數(shù)大于1。
1).準(zhǔn)備數(shù)據(jù)
--session 1: insert data
bloatdb=# delete FROM tbl;
DELETE 3000000
bloatdb=# INSERT INTO tbl VALUES(generate_series(1,100000), 'first'||(random()*(10^3))::integer, 'second'||(random()*(10^3))::integer);
INSERT 0 100000
bloatdb=# update tbl set first='chris';
UPDATE 100000
bloatdb=#
-- session 2:check bloat
[postgres@localhost ~]$ /home/soft/pg_bloat_check-master/pg_bloat_check.py -c "dbname=bloatdb" -t tbl
1. public.tbl_pkey..............................................................(41.14%) 28 MB wasted
2. public.idx_tbl_second.......................................................(4.32%) 4471 kB wasted
3. public.idx_tbl_first........................................................(2.96%) 2889 kB wasted
[postgres@localhost ~]$
2).online insert and repack
--session 1: insert large data
bloatdb=# INSERT INTO tbl VALUES(generate_series(100001,3000000), 'first'||(random()*(10^3))::integer, 'second'||(random()*(10^3))::integer);
光標(biāo)閃爍
-- session 2:process bloat,during session 1 inert large data
[postgres@localhost ~]$ pg_repack -d bloatdb --table tbl --only-indexes --wait-timeout=3600
INFO: repacking indexes of "tbl"
INFO: repacking index "public"."idx_tbl_first"
INFO: repacking index "public"."idx_tbl_second"
光標(biāo)閃爍
--session 1:insert finish
bloatdb=# INSERT INTO tbl VALUES(generate_series(100001,3000000), 'first'||(random()*(10^3))::integer, 'second'||(random()*(10^3))::integer);
INSERT 0 2900000
bloatdb=#
--session 2:repack finish
[postgres@localhost ~]$ pg_repack -d bloatdb --table tbl --only-indexes -T 3600
INFO: repacking indexes of "tbl"
INFO: repacking index "public"."idx_tbl_first"
INFO: repacking index "public"."idx_tbl_second"
INFO: repacking index "public"."tbl_pkey"
3) check table data and index bloat
--session 2:check bloat
[postgres@localhost ~]$ /home/soft/pg_bloat_check-master/pg_bloat_check.py -c "dbname=bloatdb" -t tbl
1. public.tbl_pkey..............................................................(0.0%) 0 bytes wasted
2. public.idx_tbl_first.........................................................(0.0%) 0 bytes wasted
3. public.idx_tbl_second........................................................(0.0%) 0 bytes wasted
[postgres@localhost ~]$
--session 1:check table data
bloatdb=# select count(*) from tbl;
count
---------
3000000
(1 row)
bloatdb=#
3.指定tbl表指定索引膨脹處理
注意:--index(默認(rèn)使用concurrently方式創(chuàng)建指定索引),無法與--only-indexes選項同時使用。
[postgres@localhost ~]$ pg_repack -d bloatdb --index idx_tbl_first --only-indexes
ERROR: cannot specify --index (-i) and --only-indexes (-x)
1).準(zhǔn)備數(shù)據(jù)
-- read data
bloatdb=# delete FROM tbl;
DELETE 3000000
bloatdb=# INSERT INTO tbl VALUES(generate_series(1,100000), 'first'||(random()*(10^3))::integer, 'second'||(random()*(10^3))::integer);
INSERT 0 100000
bloatdb=# update tbl set first='chris';
UPDATE 100000
bloatdb=#
-- check bloat
[postgres@localhost ~]$ /home/soft/pg_bloat_check-master/pg_bloat_check.py -c "dbname=bloatdb" -t tbl
1. public.idx_tbl_second........................................................(47.57%) 97 MB wasted
2. public.tbl_pkey.............................................................(9.44%) 7206 kB wasted
3. public.idx_tbl_first........................................................(3.11%) 3040 kB wasted
[postgres@localhost ~]$
2).online insert and repack
--session 1: insert large data
bloatdb=# INSERT INTO tbl VALUES(generate_series(100001,3000000), 'first'||(random()*(10^3))::integer, 'second'||(random()*(10^3))::integer);
光標(biāo)閃爍
-- session 2:process bloat,during session 1 inert large data
[postgres@localhost ~]$ pg_repack -d bloatdb --index idx_tbl_second --wait-timeout=3600
INFO: repacking index "public"."idx_tbl_second"
光標(biāo)閃爍
--session 1:insert finish
bloatdb=# INSERT INTO tbl VALUES(generate_series(100001,3000000), 'first'||(random()*(10^3))::integer, 'second'||(random()*(10^3))::integer);
INSERT 0 2900000
bloatdb=#
--session 2:repack finish
[postgres@localhost ~]$ pg_repack -d bloatdb --index idx_tbl_second --wait-timeout=3600
INFO: repacking index "public"."idx_tbl_second"
[postgres@localhost ~]$
3) check table data and index bloat
--session 2:check bloat
[postgres@localhost ~]$ /home/soft/pg_bloat_check-master/pg_bloat_check.py -c "dbname=bloatdb" -t tbl
1. public.idx_tbl_first........................................................(50.77%) 102 MB wasted
2. public.tbl_pkey...............................................................(47.6%) 65 MB wasted
3. public.idx_tbl_second........................................................(0.0%) 0 bytes wasted
[postgres@localhost ~]$
--session 1:check table data
bloatdb=# select count(*) from tbl;
count
---------
3000000
(1 row)
bloatdb=#
測試結(jié)論:
一般同等條件下,索引比數(shù)據(jù)更容易膨脹。
在磁盤空間較緊張的情況下,建議一條接著一條索引處理。
一般bloat處理所需磁盤空閑空間是對象size的2倍,所以處理前必須先關(guān)注空閑磁盤空間大小。
注意pg_repack版本對Pg版本的支持情況,9.6截至2016-11-26仍未支持,詳見http://pgxn.org/dist/pg_repack/doc/pg_repack.html#Releases。
處理存在在線交易的表或者索引對象的bloat時,注意設(shè)置超時參數(shù)--wait-timeout,一般設(shè)置為1800或3600(特別感謝李海龍建議)。
特別聲明:本說明只針對此次測試環(huán)境,在生產(chǎn)環(huán)境要在業(yè)務(wù)低峰時期運行,為了保證系統(tǒng)數(shù)據(jù)安全,建議先備份數(shù)據(jù),然后做膨脹處理