這篇文章將為大家詳細(xì)講解有關(guān)如何下載POSTGRESQL源碼安裝包及實(shí)現(xiàn)主機(jī)配置,小編覺(jué)得挺實(shí)用的,因此分享給大家做個(gè)參考,希望大家閱讀完這篇文章后可以有所收獲。
成都創(chuàng)新互聯(lián)-專(zhuān)業(yè)網(wǎng)站定制、快速模板網(wǎng)站建設(shè)、高性?xún)r(jià)比北屯網(wǎng)站開(kāi)發(fā)、企業(yè)建站全套包干低至880元,成熟完善的模板庫(kù),直接使用。一站式北屯網(wǎng)站制作公司更省心,省錢(qián),快速模板網(wǎng)站建設(shè)找我們,業(yè)務(wù)覆蓋北屯地區(qū)。費(fèi)用合理售后完善,十年實(shí)體公司更值得信賴(lài)。
一、下載POSTGRESQL源碼安裝包及主機(jī)配置
https://www.postgresql.org/ftp/source/v10.3/
postgresql-10.3.tar.gz
虛擬機(jī)環(huán)境
node1 192.168.159.151
node2 192.168.159.152
操作系統(tǒng)為redhat6.5
數(shù)據(jù)庫(kù)為postgresql10.3
兩個(gè)節(jié)點(diǎn)均配置/etc/hosts
vi /etc/hosts
node1 192.168.159.151
node2 192.168.159.152
二、編譯安裝
(1)創(chuàng)建postgres用戶(hù)
useradd -m -r -s /bin/bash -u 5432 postgres
(2)安裝相關(guān)依賴(lài)包
yum install gettext gcc make perl python perl-ExtUtils-Embed readline-devel zlib-devel openssl-devel libxml2-devel cmake gcc-c++ libxslt-devel openldap-devel pam-devel python-devel cyrus-sasl-devel libgcrypt-devel libgpg-error-devel libstdc++-devel
(3)配置POSTGRES
./configure --prefix=/opt/postgresql-10.3 --with-segsize=8 --with-wal-segsize=64 --with-wal-blocksize=16 --with-blocksize=16 --with-libedit-preferred --with-perl --with-python --with-openssl --with-libxml --with-libxslt --enable-thread-safety --enable-nls=zh_CN
最后幾行出現(xiàn)以下黃色輸出即配置正確,否則根據(jù)報(bào)錯(cuò)提示繼續(xù)安裝依賴(lài)包
configure: using CPPFLAGS= -D_GNU_SOURCE -I/usr/include/libxml2
configure: using LDFLAGS= -Wl,--as-needed
configure: creating ./config.status
config.status: creating GNUmakefile
config.status: creating src/Makefile.global
config.status: creating src/include/pg_config.h
config.status: creating src/include/pg_config_ext.h
config.status: creating src/interfaces/ecpg/include/ecpg_config.h
config.status: linking src/backend/port/tas/dummy.s to src/backend/port/tas.s
config.status: linking src/backend/port/dynloader/linux.c to src/backend/port/dynloader.c
config.status: linking src/backend/port/posix_sema.c to src/backend/port/pg_sema.c
config.status: linking src/backend/port/sysv_shmem.c to src/backend/port/pg_shmem.c
config.status: linking src/backend/port/dynloader/linux.h to src/include/dynloader.h
config.status: linking src/include/port/linux.h to src/include/pg_config_os.h
config.status: linking src/makefiles/Makefile.linux to src/Makefile.port
(4)編譯
make && make install
最后幾行出現(xiàn)以下黃色輸出即配置正確
make[1]: Leaving directory `/opt/postgresql-10.3/src'
make -C config install
make[1]: Entering directory `/opt/postgresql-10.3/config'
/bin/mkdir -p '/opt/postgresql-10.3/lib/pgxs/config'
/usr/bin/install -c -m 755 ./install-sh '/opt/postgresql-10.3/lib/pgxs/config/install-sh'
/usr/bin/install -c -m 755 ./missing '/opt/postgresql-10.3/lib/pgxs/config/missing'
make[1]: Leaving directory `/opt/postgresql-10.3/config'
PostgreSQL installation complete.
(5)安裝
make world && make install -world
最后幾行出現(xiàn)以下黃色輸出即配置正確
make[1]: Leaving directory `/opt/postgresql-10.3/src'
make -C config install
make[1]: Entering directory `/opt/postgresql-10.3/config'
/bin/mkdir -p '/opt/postgresql-10.3/lib/pgxs/config'
/usr/bin/install -c -m 755 ./install-sh '/opt/postgresql-10.3/lib/pgxs/config/install-sh'
/usr/bin/install -c -m 755 ./missing '/opt/postgresql-10.3/lib/pgxs/config/missing'
make[1]: Leaving directory `/opt/postgresql-10.3/config'
PostgreSQL installation complete.
make: Leaving directory `/opt/postgresql-10.3'
(6)創(chuàng)建相關(guān)目錄及配置環(huán)境變量
mkdir -p /data/pgdata/serverlog
mkdir /data/pg
su - postgres
vi .bash_profile (刪除原來(lái)的所有,以下黃色部分直接復(fù)制粘貼)
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin
export PATH
# postgres
#PostgreSQL端口
PGPORT=5432
#PostgreSQL數(shù)據(jù)目錄
PGDATA=/data/pgdata
export PGPORT PGDATA
#所使用的語(yǔ)言
export LANG=zh_CN.utf8
#PostgreSQL 安裝目錄
export PGHOME=/data/pg
#PostgreSQL 連接庫(kù)文件
export LD_LIBRARY_PATH=$PGHOME/lib:/lib64:/usr/lib64:/usr/local/lib64:/lib:/usr/lib:/usr/local/lib:$LD_LIBRARY_PATH
export DATE=`date +"%Y%m%d%H%M"`
#將PostgreSQL的命令行添加到 PATH 環(huán)境變量
export PATH=$PGHOME/bin:$PATH
#PostgreSQL的 man 手冊(cè)
export MANPATH=$PGHOME/share/man:$MANPATH
#PostgreSQL的默認(rèn)用戶(hù)
export PGUSER=postgres
#PostgreSQL默認(rèn)主機(jī)地址
export PGHOST=127.0.0.1
#默認(rèn)的數(shù)據(jù)庫(kù)名
export PGDATABASE=postgres
#定義日志存放目錄
PGLOG="$PGDATA/serverlog"source .bash_profile
(7)初始化數(shù)據(jù)庫(kù)
#執(zhí)行數(shù)據(jù)庫(kù)初始化腳本
root用戶(hù)登錄
chown -R postgres.postgres /data/
su - postgres
$/opt/postgresql-10.3/bin/initdb --encoding=utf8 -D /data/pg/data
警告:為本地連接啟動(dòng)了 "trust" 認(rèn)證.
你可以通過(guò)編輯 pg_hba.conf 更改或你下次
行 initdb 時(shí)使用 -A或者--auth-local和--auth-host選項(xiàng).
Success. You can now start the database server using:
啟動(dòng)數(shù)據(jù)庫(kù)
su - postgres
/opt/postgresql-10.3/bin/pg_ctl -D /data/pg/data -l logfile start
(8)相關(guān)命令拷貝
root用戶(hù)
mkdir /data/pg/bin
cp /opt/postgresql-10.3/bin/* /data/pg/bin
chown -R postgres.postgres /data/pg/bin
三、postgresql主從搭建
1、主庫(kù)配置
(1)創(chuàng)建一個(gè)用戶(hù)復(fù)制的用戶(hù)replica
su - postgres
psql
CREATE ROLE replica login replication encrypted password 'replica';
(2)修改pg_hba.conf文件,指定replica登錄網(wǎng)絡(luò)(最后一添加)
vi /data/pg/data/pg_hba.conf
host replication replica 192.168.159.0/24 md5
host all replica 192.168.159.0/24 trust
(3)主庫(kù)配置文件修改以下幾項(xiàng),其他不變
vi
/data/pg/data/postgresql.conf
listen_addresses = '*'
wal_level = hot_standby #熱備模式
max_wal_senders= 6 #可以設(shè)置最多幾個(gè)流復(fù)制鏈接,差不多有幾個(gè)從,就設(shè)置多少
wal_keep_segments = 10240 #重要配置
wal_send_timeout = 60s
max_connections = 512 #從庫(kù)的 max_connections要大于主庫(kù)
archive_mode = on #允許歸檔
archive_command = 'cp %p /data/pg/data/archive/%f' #根據(jù)實(shí)際情況設(shè)置
checkpoint_timeout = 30min
max_wal_size = 3GB
min_wal_size = 64MB
mkdir /data/pg/data/archive
2、從庫(kù)環(huán)境
(1)把備庫(kù)的數(shù)據(jù)文件夾目錄清空
rm -rf /var/lib/pgsql/10/data/*
(2)在備庫(kù)上運(yùn)行
pg_basebackup -F p --progress -D /data/pg/data/ -h 192.168.159.151 -p 5432 -U replica --password
輸入密碼replica
?。?!注意,復(fù)制完成后,在備庫(kù)一定要將數(shù)據(jù)目錄下的所有文件重新授權(quán)
chown -R postgres.postgres /data/pg/data/
(3)創(chuàng)建recovery.conf 文件
cp /opt/postgresql-10.3/share/recovery.conf.sample /data/pg/data/recovery.conf
vi /data/pg/data/recovery.conf
standby_mode = on
primary_conninfo = 'host=192.168.159.151 port=5432 user=replica password=replica'
recovery_target_timeline = 'latest'
trigger_file = '/data/pg/data/trigger.kenyon'
(4)配置postgresql.conf文件
vi
/data/pg/data/postgresql.conf
listen_addresses ='*'
wal_level = hot_standby
max_connections =1000 #一般從的最大鏈接要大于主的
hot_standby =on #說(shuō)明這臺(tái)機(jī)器不僅僅用于數(shù)據(jù)歸檔,也用于查詢(xún)
max_standby_streaming_delay =30s
wal_receiver_status_interval = 10s #多久向主報(bào)告一次從的狀態(tài)
hot_standby_feedback = on #如果有錯(cuò)誤的數(shù)據(jù)復(fù)制,是否向主進(jìn)行范例
(5)啟動(dòng)備庫(kù)
su - postgres
/opt/postgresql-10.3/bin/pg_ctl -D /data/pg/data/ -l logfile start
如果無(wú)法啟動(dòng),到主庫(kù)復(fù)制文件postmaster.opts到備庫(kù)如下操作:
scp /data/pg/data/postmaster.opts 192.168.159.152:/data/pg/data/
chown -R postgres.postgres
/data/pg/data/
cd /data/pg/
chmod 700 data/
3、驗(yàn)證主從功能
主庫(kù)查詢(xún)
su - postgres
psql
postgres=# select client_addr,sync_state from pg_stat_replication;
client_addr | sync_state
-----------------+------------
192.168.159.152 | async
(1 row)
發(fā)現(xiàn)登陸postgres時(shí)出現(xiàn)以下問(wèn)題
-bash-4.1$
root用戶(hù)執(zhí)行
cp /etc/skel/.bash* /var/lib/pgsql/
再次登陸即可變成
[postgres@node1 ~]$4、手動(dòng)主備切換
(1)創(chuàng)建備庫(kù)recovery.conf 文件( 在備庫(kù)上操作192.168.159.152)
cp $PGHOME/share/recovery.conf.sample $PGDATA/recovery.conf
配置以下參數(shù)
standby_mode = 'on' --標(biāo)記PG為STANDBY SERVER
primary_conninfo = 'host=192.168.159.151 port=5432 user=replica password=replica' --標(biāo)識(shí)主庫(kù)信息
trigger_file = '/data/pg/data/trigger.kenyon' --標(biāo)識(shí)觸發(fā)器文件
(2) 關(guān)閉主庫(kù)(在主庫(kù)上操作192.168.159.151)
/opt/postgresql-10.3/bin/pg_ctl -D /data/pg/data/ -l logfile stop
(3)激活備庫(kù)到主庫(kù)狀態(tài) ( 在備庫(kù)上操作192.168.159.152 )
激活備庫(kù)只要?jiǎng)?chuàng)建一個(gè)文件即可,根據(jù)備庫(kù) recovery.conf 配置文件的參數(shù) trigger_file 值,
創(chuàng)建這個(gè) trigger 文件即可。 例如 "touch /data/pg/data/trigger.kenyon"
touch /data/pg/data/trigger.kenyon
過(guò)一會(huì)兒發(fā)現(xiàn) recovery.conf 文件變成 recovery.done ,說(shuō)明備庫(kù)已經(jīng)激活。
查看logfile日志,出現(xiàn)以下信息即激活
2018-06-04 21:11:01.137 PDT [12818] 日志: 已找到觸發(fā)器文件:/data/pg/data/trigger.kenyon
2018-06-04 21:11:01.148 PDT [12818] 日志: redo 在 0/C02A390 完成
2018-06-04 21:11:01.172 PDT [12818] 日志: 已選擇的新時(shí)間線ID:2
2018-06-04 21:11:05.442 PDT [12818] 日志: 歸檔恢復(fù)完畢
2018-06-04 21:11:05.568 PDT [12817] 日志: 數(shù)據(jù)庫(kù)系統(tǒng)準(zhǔn)備接受連接
(4)修改原來(lái)主庫(kù)的配置文件
vi /data/pg/data/postgresql.conf
max_connections = 1500 #從庫(kù)值要比主庫(kù)值大
(5)激活原來(lái)的主庫(kù),讓其轉(zhuǎn)變成從庫(kù) (在原來(lái)的主庫(kù)上執(zhí)行192.168.159.151)
--創(chuàng)建 $PGDATA/recovery.conf 文件,配置以下參數(shù)
vi /data/pg/data/recovery.conf
recovery_target_timeline = 'latest'
standby_mode = 'on' #--標(biāo)記PG為STANDBY SERVER
primary_conninfo = 'host=192.168.159.152 port=5432 user=replica password=replica' #--標(biāo)識(shí)主庫(kù)信息
trigger_file = '/data/pg/data/trigger.kenyon' #--標(biāo)識(shí)觸發(fā)器文件
--修改 pg_hba.conf (現(xiàn)在的主庫(kù)上增加192.168.159.152),添加以下配置
vi /data/pg/data/pg_hba.conf
host replication replica 192.168.159.151/32 md5
啟動(dòng)原來(lái)的主庫(kù)即現(xiàn)在的從庫(kù)(192.168.159.151)
/opt/postgresql-10.3/bin/pg_ctl -D /data/pg/data/ -l logfile start
查看現(xiàn)在的從庫(kù)logfile日志發(fā)現(xiàn)報(bào)錯(cuò)信息
2018-06-05 00:08:00.326 PDT [9729] 詳細(xì)信息: WAL結(jié)束時(shí),到了時(shí)間線1和地址0/C02A400.
2018-06-05 00:08:00.327 PDT [9725] 日志: 在當(dāng)前恢復(fù)點(diǎn)0/C02A630之前, 新的時(shí)間點(diǎn)2脫離了當(dāng)前茅的數(shù)據(jù)庫(kù)系統(tǒng)時(shí)間點(diǎn)1
2018-06-05 00:08:05.322 PDT [9729] 日志: 在0/C000000處時(shí)間線1上重啟WAL流操作
2018-06-05 00:08:05.327 PDT [9729] 日志: 復(fù)制由主用服務(wù)器終止
2018-06-05 00:08:05.327 PDT [9729] 詳細(xì)信息: WAL結(jié)束時(shí),到了時(shí)間線1和地址0/C02A400.
2018-06-05 00:08:05.329 PDT [9725] 日志: 在當(dāng)前恢復(fù)點(diǎn)0/C02A630之前, 新的時(shí)間點(diǎn)2脫離了當(dāng)前茅的數(shù)據(jù)庫(kù)系統(tǒng)時(shí)間點(diǎn)1
2018-06-05 00:08:10.328 PDT [9729] 日志: 在0/C000000處時(shí)間線1上重啟WAL流操作
2018-06-05 00:08:10.332 PDT [9729] 日志: 復(fù)制由主用服務(wù)器終止
2018-06-05 00:08:10.332 PDT [9729] 詳細(xì)信息: WAL結(jié)束時(shí),到了時(shí)間線1和地址0/C02A400.
2018-06-05 00:08:10.333 PDT [9725] 日志: 在當(dāng)前恢復(fù)點(diǎn)0/C02A630之前, 新的時(shí)間點(diǎn)2脫離了當(dāng)前茅的數(shù)據(jù)庫(kù)系統(tǒng)時(shí)間點(diǎn)1
在現(xiàn)在的主庫(kù)操作:
scp
/data/pg/data/pg_wal/00000002.history 192.168.159.151:/data/pg/data/pg_wal/
(6)兩個(gè)節(jié)點(diǎn)都操作
vi /data/pg/data/recovery.conf
restore_command = 'cp /data/pg/data/archive/%f %p'
mkdir /data/pg/data/archive
chown postgres.postgres /data/pg/data/archive
vi /data/pg/data/postgresql.conf
archive_command = 'cp %p /data/pg/data/archive/%f'
四、安裝PGPOOL
(1)配置兩臺(tái)機(jī)器的ssh免密鑰登錄
1節(jié)點(diǎn)
[postgres@node1]$ ssh-keygen -t rsa
全部回車(chē)默認(rèn)
[postgres@node1]$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
[postgres@node1]$ chmod go-rwx ~/.ssh/*
[postgres@node1]$ cd ~/.ssh
2節(jié)點(diǎn)
[postgres@node2$ ssh-keygen -t rsa
全部回車(chē)默認(rèn)
[postgres@node2$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
[postgres@node2$ chmod go-rwx ~/.ssh/*
[postgres@node2] cd ~/.ssh
1節(jié)點(diǎn)
[postgres@node1]$ scp id_rsa.pub 192.168.159.152:/home/postgres/.ssh/id_rsa.pub1
2節(jié)點(diǎn)
[postgres@node2] cat id_rsa.pub1 >> authorized_keys
[postgres@node2]scp id_rsa.pub 192.168.159.151:/home/postgres/.ssh/id_rsa.pub2
1節(jié)點(diǎn)
[postgres@node1] cat id_rsa.pub2 >> authorized_keys
(2)安裝pgpool ii
安裝pgpool ii
yum -y install libmemcached postgresql-libs.x86_64 openssl098e
(這里注意一定要先安裝這些YUM源,不然死活安裝不了pgpool II)
rpm -ivh pgpool-II-pg10-3.7.2-1pgdg.rhel6.x86_64.rpm
pg_md5 -u postgres -p
密碼設(shè)置為postgres
輸出的密碼編碼為
e8a48653851e28c69d0506508fb27fc5
vi /etc/pgpool-II/pcp.conf #最后一行添加
postgres:e8a48653851e28c69d0506508fb27fc5
mkdir -p /opt/pgpool/oiddir
cp /etc/pgpool-II/pgpool.conf /etc/pgpool-II/pgpool.conf.bak
ifconfig查看下網(wǎng)卡
[root@node1 pgpool-II]# ifconfig
eth2 Link encap:Ethernet HWaddr 00:0C:29:9E:E8:6D
inet addr:192.168.159.152 Bcast:192.168.159.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe9e:e86d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:14557 errors:0 dropped:0 overruns:0 frame:0
TX packets:10820 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1889055 (1.8 MiB) TX bytes:1485329 (1.4 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:5029 errors:0 dropped:0 overruns:0 frame:0
TX packets:5029 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2786891 (2.6 MiB) TX bytes:2786891 (2.6 MiB)
注意:這里我的網(wǎng)卡是eth2,所以,下面2個(gè)節(jié)點(diǎn)的黃色字體配置要相應(yīng)變化
如果網(wǎng)卡配置不正確,會(huì)報(bào)類(lèi)似這樣的報(bào)錯(cuò)arping: unknown iface eth0
1節(jié)點(diǎn)
vi /etc/pgpool-II/pgpool.conf
listen_addresses = '*'
port = 9999
socket_dir = '/opt/pgpool'
pcp_port = 9898
pcp_socket_dir = '/opt/pgpool'
backend_hostname0 = '192.168.159.151' ##配置數(shù)據(jù)節(jié)點(diǎn) node1
backend_port0 = 5432
backend_weight0 = 1
backend_flag0 = 'ALLOW_TO_FAILOVER'
backend_hostname1 = '192.168.159.152' ##配置數(shù)據(jù)節(jié)點(diǎn) node2
backend_port1 = 5432
backend_weight1 = 1
backend_flag1 = 'ALLOW_TO_FAILOVER'
enable_pool_hba = on
pool_passwd = 'pool_passwd'
authentication_timeout = 60
ssl = off
num_init_children = 32
max_pool = 4
child_life_time = 300
child_max_connections = 0
connection_life_time = 0
client_idle_limit = 0
log_destination = 'syslog'
print_timestamp = on
log_connections = on
log_hostname = on
log_statement = on
log_per_node_statement = off
log_standby_delay = 'none'
syslog_facility = 'LOCAL0'
syslog_ident = 'pgpool'
debug_level = 0
pid_file_name = '/opt/pgpool/pgpool.pid'
logdir = '/tmp'
connection_cache = on
reset_query_list = 'ABORT; DISCARD ALL'
replication_mode = off
replicate_select = off
insert_lock = on
lobj_lock_table = ''
replication_stop_on_mismatch = off
failover_if_affected_tuples_mismatch = off
load_balance_mode = on
ignore_leading_white_space = on
white_function_list = ''
black_function_list = 'nextval,setval'
master_slave_mode = on # 設(shè)置流復(fù)制模式
master_slave_sub_mode = 'stream' # 設(shè)置流復(fù)制模式
sr_check_period = 5
sr_check_user = 'replica'
sr_check_password = 'replica'
delay_threshold = 16000
follow_master_command = ''
parallel_mode = off
pgpool2_hostname = ''
system_db_hostname = 'localhost'
system_db_port = 5432
system_db_dbname = 'pgpool'
system_db_schema = 'pgpool_catalog'
system_db_user = 'pgpool'
system_db_password = ''
health_check_period = 5
health_check_timeout = 20
health_check_user = 'replica'
health_check_password = 'replcia'
health_check_max_retries = 3
health_check_retry_delay = 1
failover_command = '/opt/pgpool/failover_stream.sh %d %H
/data/pg/data/trigger.kenyon'
failback_command = ''
fail_over_on_backend_error = on
search_primary_node_timeout = 10
recovery_user = 'nobody'
recovery_password = ''
recovery_1st_stage_command = ''
recovery_2nd_stage_command = ''
recovery_timeout = 90
client_idle_limit_in_recovery = 0
use_watchdog = on
trusted_servers = ''
ping_path = '/bin'
wd_hostname = '192.168.159.151'
wd_port = 9000
wd_authkey = ''
delegate_IP = '192.168.159.153'
ifconfig_path = '/sbin'
if_up_cmd = 'ifconfig
eth2:0
inet $_IP_$ netmask 255.255.255.0'
if_down_cmd = 'ifconfig
eth2:0
down'
arping_path = '/usr/sbin' # arping command path
arping_cmd = 'arping -I eth2 -U $_IP_$ -w 1' #-I eth2指定出口網(wǎng)卡
clear_memqcache_on_escalation = on
wd_escalation_command = ''
wd_lifecheck_method = 'heartbeat'
wd_interval = 10
wd_heartbeat_port = 9694
wd_heartbeat_keepalive = 2
wd_heartbeat_deadtime = 30
heartbeat_destination0 = '192.168.159.152' # 配置對(duì)端的 hostname
heartbeat_destination_port0 = 9694
heartbeat_device0 =
'eth2'
wd_life_point = 3
wd_lifecheck_query = 'SELECT 1'
wd_lifecheck_dbname = 'template1'
wd_lifecheck_user = 'nobody'
wd_lifecheck_password = ''
other_pgpool_hostname0 = '192.168.159.152' ## 配置對(duì)端的 pgpool
other_pgpool_port0 = 9999
other_wd_port0 = 9000
relcache_expire = 0
relcache_size = 256
check_temp_table = on
memory_cache_enabled = off
memqcache_method = 'shmem'
memqcache_memcached_host = 'localhost'
memqcache_memcached_port = 11211
memqcache_total_size = 67108864
memqcache_max_num_cache = 1000000
memqcache_expire = 0
memqcache_auto_cache_invalidation = on
memqcache_maxcache = 409600
memqcache_cache_block_size = 1048576
memqcache_oiddir = '/opt/pgpool/oiddir'#(需要現(xiàn)在/opt/pgpool目錄下創(chuàng)建oiddr)
white_memqcache_table_list = ''
black_memqcache_table_list = ''
2節(jié)點(diǎn)
vi /etc/pgpool-II/pgpool.conf
listen_addresses = '*'
port = 9999
socket_dir = '/opt/pgpool'
pcp_port = 9898
pcp_socket_dir = '/opt/pgpool'
backend_hostname0 = '192.168.159.151'
backend_port0 = 5432
backend_weight0 = 1
backend_flag0 = 'ALLOW_TO_FAILOVER'
backend_hostname1 = '192.168.159.152'
backend_port1 = 5432
backend_weight1 = 1
backend_flag1 = 'ALLOW_TO_FAILOVER'
enable_pool_hba = on
pool_passwd = 'pool_passwd'
authentication_timeout = 60
ssl = off
num_init_children = 32
max_pool = 4
child_life_time = 300
child_max_connections = 0
connection_life_time = 0
client_idle_limit = 0
log_destination = 'syslog'
print_timestamp = on
log_connections = on
log_hostname = on
log_statement = on
log_per_node_statement = off
log_standby_delay = 'none'
syslog_facility = 'LOCAL0'
syslog_ident = 'pgpool'
debug_level = 0
pid_file_name = '/opt/pgpool/pgpool.pid'
logdir = '/tmp'
connection_cache = on
reset_query_list = 'ABORT; DISCARD ALL'
replication_mode = off
replicate_select = off
insert_lock = on
lobj_lock_table = ''
replication_stop_on_mismatch = off
failover_if_affected_tuples_mismatch = off
load_balance_mode = on
ignore_leading_white_space = on
white_function_list = ''
black_function_list = 'nextval,setval'
master_slave_mode = on
master_slave_sub_mode = 'stream'
sr_check_period = 0
sr_check_user = 'replica'
sr_check_password = 'replica'
delay_threshold = 16000
follow_master_command = ''
parallel_mode = off
pgpool2_hostname = ''
system_db_hostname = 'localhost'
system_db_port = 5432
system_db_dbname = 'pgpool'
system_db_schema = 'pgpool_catalog'
system_db_user = 'pgpool'
system_db_password = ''
health_check_period = 0
health_check_timeout = 20
health_check_user = 'nobody'
health_check_password = ''
health_check_max_retries = 0
health_check_retry_delay = 1
failover_command = '/opt/pgpool/failover_stream.sh %d %H /file/data/trigger/file'
failback_command = ''
fail_over_on_backend_error = on
search_primary_node_timeout = 10
recovery_user = 'nobody'
recovery_password = ''
recovery_1st_stage_command = ''
recovery_2nd_stage_command = ''
recovery_timeout = 90
client_idle_limit_in_recovery = 0
use_watchdog = off
trusted_servers = ''
ping_path = '/bin'
wd_hostname = ' '
wd_port = 9000
wd_authkey = ''
delegate_IP = '192.168.159.153'
ifconfig_path = '/sbin'
if_up_cmd = 'ifconfig
eth2:0
inet $_IP_$ netmask 255.255.255.0'
if_down_cmd = 'ifconfig
eth2:0
down'
arping_path = '/usr/sbin' # arping command path
arping_cmd = 'arping -I eth2 -U $_IP_$ -w 1' #-I eth2指定出口網(wǎng)卡
clear_memqcache_on_escalation = on
wd_escalation_command = ''
wd_lifecheck_method = 'heartbeat'
wd_interval = 10
wd_heartbeat_port = 9694
wd_heartbeat_keepalive = 2
wd_heartbeat_deadtime = 30
heartbeat_destination0 = '192.168.159.151'
heartbeat_destination_port0 = 9694
heartbeat_device0 = 'eth2'
wd_life_point = 3
wd_lifecheck_query = 'SELECT 1'
wd_lifecheck_dbname = 'template1'
wd_lifecheck_user = 'nobody'
wd_lifecheck_password = ''
other_pgpool_hostname0 = '192.168.159.152'
other_pgpool_port0 = 9999
other_wd_port0 = 9000
relcache_expire = 0
relcache_size = 256
check_temp_table = on
memory_cache_enabled = off
memqcache_method = 'shmem'
memqcache_memcached_host = 'localhost'
memqcache_memcached_port = 11211
memqcache_total_size = 67108864
memqcache_max_num_cache = 1000000
memqcache_expire = 0
memqcache_auto_cache_invalidation = on
memqcache_maxcache = 409600
memqcache_cache_block_size = 1048576
memqcache_oiddir = '/opt/pgpool/oiddir'
white_memqcache_table_list = ''
black_memqcache_table_list = ''
vi /opt/pgpool/failover_stream.sh
#! /bin/sh
# Failover command for streaming replication.
# This script assumes that DB node 0 is primary, and 1 is standby.
#
# If standby goes down, do nothing. If primary goes down, create a
# trigger file so that standby takes over primary node.
#
# Arguments: $1: failed node id. $2: new master hostname. $3: path to
# trigger file.
failed_node=$1
new_master=$2
trigger_file=$3
# Do nothing if standby goes down.
#if [ $failed_node = 1 ]; then
# exit 0;
#fi
/usr/bin/ssh -T $new_master /bin/touch $trigger_file
exit 0;
給腳本授權(quán)
chmod u+x /opt/pgpool/failover_stream.sh
scp /opt/pgpool/failover_stream.sh 192.168.159.152:/opt/pgpool/
cp /etc/pgpool-II/pool_hba.conf /etc/pgpool-II/pool_hba.conf.bak
vi /etc/pgpool-II/pool_hba.conf
host all all 192.168.159.151/32 trust
host replication replica 192.168.159.151/32 trust
host postgres postgres 192.168.159.151/32 trust
host all all 192.168.159.152/32 trust
host replication replica 192.168.159.152/32 trust
host postgres postgres 192.168.159.152/32 trust
host postgres postgres 192.168.159.152/32 trust
host all all 192.168.159.153/32 trust
host replication replica 192.168.159.153/32 trust
host postgres postgres 192.168.159.153/32 trust
host postgres postgres 192.168.159.153/32 trust
注意192.168.159.153 是VIP地址
scp /etc/pgpool-II/pool_hba.conf 192.168.159.140:/etc/pgpool-II/
啟動(dòng)pgpool
pgpool -n &
關(guān)閉pgpool
pgpool -m fast stop
登陸pgpool
/data/pg/bin/psql -h 192.168.159.151 -p 9999 -U postgres -d postgres
也可以用VIP登陸/data/pg/bin/psql -h 192.168.159.153 -p 9999 -U postgres -d postgres
查看pgpool節(jié)點(diǎn)
show pool_nodes;
postgres=# show pool_nodes;
node_id | hostname | port | status | lb_weight | role | select_cnt | load_balance_node | replication_delay
---------+-----------------+------+--------+-----------+---------+------------+-------------------+-------------------
0 | 192.168.159.151 | 5432 | up | 0.500000 | primary | 0 | true | 0
1 | 192.168.159.152 | 5432 | down | 0.500000 | standby | 0 | false | 0
(2 rows)
五、安裝keepalive
tar xvf keepalived-1.4.2.tar.gz
cd keepalived-1.4.2
./configure
make
make install
mkdir /etc/keepalived
cd /etc/keepalived/
1節(jié)點(diǎn):
vi
/etc/keepalived/keepalived.conf
global_defs {
router_id node1
}
vrrp_instance VI_1 {
state BACKUP #設(shè)置為主服務(wù)器
interface eth2:0 #監(jiān)測(cè)網(wǎng)絡(luò)接口
virtual_router_id 51 #主、備必須一樣
priority 100 #(主、備機(jī)取不同的優(yōu)先級(jí),主機(jī)值較大,備份機(jī)值較小,值越大優(yōu)先級(jí)越高)
advert_int 1 #VRRP Multicast廣播周期秒數(shù)
authentication {
auth_type PASS #VRRP認(rèn)證方式,主備必須一致
auth_pass 1111 #(密碼)
}
virtual_ipaddress {
192.168.159.153/24 #VRRP HA虛擬地址
}
2節(jié)點(diǎn):
vi /etc/keepalived/keepalived.conf
global_defs {
router_id node2
}
vrrp_instance VI_1 {
state BACKUP #設(shè)置為主服務(wù)器
interface eth2:0 #監(jiān)測(cè)網(wǎng)絡(luò)接口
virtual_router_id 51 #主、備必須一樣
priority 90 #(主、備機(jī)取不同的優(yōu)先級(jí),主機(jī)值較大,備份機(jī)值較小,值越大優(yōu)先級(jí)越高)
advert_int 1 #VRRP Multicast廣播周期秒數(shù)
authentication {
auth_type PASS #VRRP認(rèn)證方式,主備必須一致
auth_pass 1111 #(密碼)
}
virtual_ipaddress {
192.168.159.153/24 #VRRP HA虛擬地址
}
啟動(dòng)Keepalived
keepalived -D -f /etc/keepalived/keepalived.conf
查看日志
tail -f /var/log/message
查看進(jìn)程
ps -ef|grep keepalive
?。。。?!注意?。。。。。。? 配置PGPOOL的高可用,以下內(nèi)容為本人親測(cè),部分關(guān)鍵性資料是自己摸索編寫(xiě)的,網(wǎng)上找不到資料
1、設(shè)置相關(guān)權(quán)限(兩個(gè)節(jié)點(diǎn)都要執(zhí)行)
--配置 ifconfig, arping 執(zhí)行權(quán)限 root用戶(hù)下執(zhí)行
chmod u+s /sbin/ifconfig
chmod u+s /sbin/ifdown
chmod u+s /sbin/ifup
chmod u+s /usr/sbin/
chmod 755 /opt/pgpool/failover_stream.sh
chown postgres.root /opt/pgpool/failover_stream.sh
2、配置PGPOOL日志(兩個(gè)節(jié)點(diǎn)都要執(zhí)行)最后一行添加
vi /etc/rsyslog.conf
local0.* /var/log/pgpool.log
/etc/init.d/rsyslog restart
3、配置關(guān)鍵腳本failover_stream.sh(兩個(gè)節(jié)點(diǎn)都要執(zhí)行)
將原來(lái)的那個(gè)ssh那行刪除或注釋
主庫(kù)是192.168.159.151時(shí)
vi /opt/pgpool/failover_stream.sh
ifconfig eth2:0 down
/usr/bin/ssh 192.168.159.152 /bin/touch /data/pg/data/trigger.kenyon
/usr/bin/ssh 192.168.159.152 ifconfig eth2:0 up
主庫(kù)是192.168.159.152時(shí)
vi /opt/pgpool/failover_stream.sh
ifconfig eth2:0 down
/usr/bin/ssh 192.168.159.151 /bin/touch /data/pg/data/trigger.kenyon
/usr/bin/ssh 192.168.159.151 ifconfig eth2:0 up
4、復(fù)制一個(gè)eth2:0的網(wǎng)卡(兩個(gè)節(jié)點(diǎn)都要執(zhí)行)
cd /etc/sysconfig/network-scripts/
cp ifcfg-eth2 ifcfg-eth2:0
vi ifcfg-eth2:0
DEVICE="eth2:0"
BOOTPROTO="static"
HWADDR="00:0c:29:0c:7d:4f"
IPV6INIT="yes"
NM_CONTROLLED="yes"
ONBOOT="yes"
TYPE="Ethernet"
#UUID="e618ec6a-8bb0-4202-8fe6-54febd0f8c76"
IPADDR=192.168.159.153
NETMASK=255.255.255.0
GATEWAY=192.168.159.1
5、修改pgpool.conf配置文件
vi /etc/pgpool-II/pgpool.conf
failover_command = '/opt/pgpool/failover_stream.sh'
將原來(lái)的那行注釋掉,用這種方式就可以了
6、修改 pgpool.conf配置文件
vi /etc/pgpool-II/pgpool.conf
heartbeat_device0 = 'eth2:0'
在本文上面的第三大點(diǎn)的第四小點(diǎn)有手動(dòng)主備切換的步驟
VIP暫時(shí)還是不能自動(dòng)漂浮切換,但是可以手動(dòng)切換主備(按上面的配置會(huì)出現(xiàn)兩個(gè)節(jié)點(diǎn)都有VIP192.168.159.153,這就很奇怪)
目前手動(dòng)切換主備,可以實(shí)現(xiàn)VIP自動(dòng)漂浮切換,但是前提條件是pgpool必須停止,比如主節(jié)點(diǎn)的postgresql數(shù)據(jù)庫(kù)實(shí)例停止了,那同時(shí)主節(jié)點(diǎn)pgpool也要停止,這樣過(guò)幾分鐘左右,從節(jié)點(diǎn)的vip192.168.159.153就會(huì)自動(dòng)起來(lái)。
一定要注意的是,主備切換動(dòng)作完成后,要用命令查看及測(cè)試切換是否成功,比如登陸
/data/pg/bin/psql -h 192.168.159.153 -p 9999 -U postgres -d postgres
show pool_nodes;
select client_addr,sync_state from pg_stat_replication;
這些查到的信息確認(rèn)正確后,再?lài)L試create一個(gè)測(cè)試table看是否能夠創(chuàng)建
create table test123 (tt int);
注意,在/data/pg/data/gprof目錄下,有很多的一些二進(jìn)制文件,不知道是什么,會(huì)占用大量的存儲(chǔ)空間。請(qǐng)教大神指導(dǎo),哈哈PG主從+pgpool ii 搭建完成后 相關(guān)報(bào)錯(cuò)
1、PG無(wú)法登陸問(wèn)題
原本第一步搭建好PG主從,測(cè)試主從同步功能,登陸都沒(méi)問(wèn)題
但是在后續(xù)安裝配置pgpool ii 高可用的時(shí)候,突然發(fā)現(xiàn)PG無(wú)法登陸了,報(bào)錯(cuò)如下:
[postgres@node1 ~]$ psql
psql: symbol lookup error: psql: undefined symbol: PQconnectdbParams
因?yàn)閱?dòng)PG的時(shí)候是有執(zhí)行日志的
[postgres@node1 ~]$ /opt/postgresql-10.3/bin/pg_ctl -D /data/pg/data -l logfile start
可以在postgres家目錄查看logfile日志報(bào)錯(cuò)信息:
2018-05-31 23:00:18.703 PDT [12734] 致命錯(cuò)誤: 無(wú)法加載庫(kù) "/opt/postgresql-10.3/lib/libpqwalreceiver.so": /opt/postgresql-10.3/lib/libpqwalreceiver.so: undefined symbol: PQescapeIdentifier
2018-05-31 23:00:23.709 PDT [12736] 致命錯(cuò)誤: 無(wú)法加載庫(kù) "/opt/postgresql-10.3/lib/libpqwalreceiver.so": /opt/postgresql-10.3/lib/libpqwalreceiver.so: undefined symbol: PQescapeIdentifier
2018-05-31 23:00:28.715 PDT [12737] 致命錯(cuò)誤: 無(wú)法加載庫(kù) "/opt/postgresql-10.3/lib/libpqwalreceiver.so": /opt/postgresql-10.3/lib/libpqwalreceiver.so: undefined symbol: PQescapeIdentifier
2018-05-31 23:00:33.721 PDT [12738] 致命錯(cuò)誤: 無(wú)法加載庫(kù) "/opt/postgresql-10.3/lib/libpqwalreceiver.so": /opt/postgresql-10.3/lib/libpqwalreceiver.so: undefined symbol: PQescapeIdentifier
2018-05-31 23:00:38.730 PDT [12739] 致命錯(cuò)誤: 無(wú)法加載庫(kù) "/opt/postgresql-10.3/lib/libpqwalreceiver.so": /opt/postgresql-10.3/lib/libpqwalreceiver.so: undefined symbol: PQescapeIdentifier
這個(gè)時(shí)候,可以臨時(shí)執(zhí)行
export LD_LIBRARY_PATH=/opt/postgresql-10.3/lib/libpqwalreceiver.so
加載缺失的庫(kù)文件,再重新啟動(dòng)PG就可以登錄了
想要永久的解決,如下:
vi ~/.bash_profile
在最后一行添加
export LD_LIBRARY_PATH=/opt/postgresql-10.3/lib/libpqwalreceiver.so2、PGPOOL無(wú)法啟動(dòng)問(wèn)題
使用命令pgpool -n & 啟動(dòng)pgpool,發(fā)現(xiàn)無(wú)法啟動(dòng)
[root@node1 ~]# ps -ef|grep pgpool
root 3163 3081 0 19:57 pts/0 00:00:00 pgpool -n
root 3205 3163 0 19:57 pts/0 00:00:00 pgpool: health check process(0)
root 3206 3163 0 19:57 pts/0 00:00:02 pgpool: health check process(1)
root 4505 4455 0 20:37 pts/1 00:00:00 grep pgpool
ps命令查看pgpool進(jìn)程,發(fā)現(xiàn)存在殘留進(jìn)程
kill 3205
kill 3206
再次啟動(dòng)pgpool成功
成功啟動(dòng)的pgpool是以下這樣的
[root@node1 ~]# ps -ef|grep pool
root 12828 2231 0 19:58 pts/0 00:00:00 pgpool -n
root 12829 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12830 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12831 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12832 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12833 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12834 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12835 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12836 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12837 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12838 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12839 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12840 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12841 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12842 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12843 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12844 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12845 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12846 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12847 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12848 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12849 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12850 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12851 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12852 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12853 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12854 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12855 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12856 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12857 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12858 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12859 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12860 12828 0 19:58 pts/0 00:00:00 pgpool: wait for connection request
root 12861 12828 0 19:58 pts/0 00:00:00 pgpool: PCP: wait for connection request
root 12862 12828 0 19:58 pts/0 00:00:00 pgpool: worker process
root 12863 12828 0 19:58 pts/0 00:00:00 pgpool: health check process(0)
root 12864 12828 0 19:58 pts/0 00:00:00 pgpool: health check process(1)
root 14061 14045 0 20:37 pts/1 00:00:00 grep pool
3、PG數(shù)據(jù)庫(kù)無(wú)法啟動(dòng)
[postgres@node2 data]$ /opt/postgresql-10.3/bin/pg_ctl -D /data/pg/data/ -l logfile start
報(bào)錯(cuò):
等待服務(wù)器進(jìn)程啟動(dòng) .... 已停止等待
pg_ctl: 無(wú)法啟動(dòng)服務(wù)器進(jìn)程
檢查日志輸出.
按報(bào)錯(cuò)提示查看日志
tail logfile
2018-05-30 22:40:05.208 PDT [16383] 日志: 在0/8000130上已到達(dá)一致性恢復(fù)狀態(tài)
2018-05-30 22:40:05.208 PDT [16382] 日志: 數(shù)據(jù)庫(kù)系統(tǒng)準(zhǔn)備接受只讀請(qǐng)求的連接
2018-05-30 22:40:05.242 PDT [16387] 日志: 在時(shí)間點(diǎn): 0/C000000 (時(shí)間安排1)啟動(dòng)日志的流操作
2018-05-30 23:19:59.272 PDT [16382] 日志: 接到到智能 (smart) 停止請(qǐng)求
2018-05-30 23:19:59.325 PDT [16387] 致命錯(cuò)誤: 由于管理員命令中斷walreceiver進(jìn)程
2018-05-30 23:19:59.332 PDT [16384] 日志: 正在關(guān)閉
2018-05-30 23:19:59.426 PDT [16382] 日志: 數(shù)據(jù)庫(kù)系統(tǒng)已關(guān)閉
2018-06-03 23:59:31.974 PDT [15817] 致命錯(cuò)誤: 無(wú)法寫(xiě)入鎖文件 "postmaster.pid": 設(shè)備上沒(méi)有空間
2018-06-04 00:00:32.287 PDT [15840] 致命錯(cuò)誤: 無(wú)法寫(xiě)入鎖文件 "postmaster.pid": 設(shè)備上沒(méi)有空間
2018-06-04 00:01:54.556 PDT [15867] 致命錯(cuò)誤: 無(wú)法寫(xiě)入鎖文件 "postmaster.pid": 設(shè)備上沒(méi)有空間
df -h查看磁盤(pán)空間,果然磁盤(pán)空間不足
[postgres@node2 data]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 18G 17G 18M 100% /
tmpfs 242M 72K 242M 1% /dev/shm
/dev/sda1 291M 39M 238M 14% /boot
[postgres@node2 data]$
4、主從切換后,從庫(kù)日志報(bào)錯(cuò)
主從切換后,從庫(kù)logfile日志報(bào)錯(cuò):
2018-07-01 21:08:41.889 PDT [2644] 日志: listening on IPv4 address "0.0.0.0", port 5432
2018-07-01 21:08:41.889 PDT [2644] 日志: listening on IPv6 address "::", port 5432
2018-07-01 21:08:41.893 PDT [2644] 日志: listening on Unix socket "/tmp/.s.PGSQL.5432"
2018-07-01 21:08:41.954 PDT [2645] 日志: 數(shù)據(jù)庫(kù)上次關(guān)閉時(shí)間為 2018-07-01 21:08:41 PDT
2018-07-01 21:08:42.008 PDT [2644] 日志: 數(shù)據(jù)庫(kù)系統(tǒng)準(zhǔn)備接受連接
從庫(kù)的安裝目錄需增加文件recovery.conf
且需配置如下:
vi /data/pg/data/recovery.conf
recovery_target_timeline = 'latest'
standby_mode = 'on' #--標(biāo)記PG為STANDBY SERVER
primary_conninfo = 'host=192.168.159.152 port=5432 user=replica password=replica' #--標(biāo)識(shí)主庫(kù)信息
trigger_file = '/data/pg/data/trigger.kenyon' #--標(biāo)識(shí)觸發(fā)器文件
關(guān)于“如何下載POSTGRESQL源碼安裝包及實(shí)現(xiàn)主機(jī)配置”這篇文章就分享到這里了,希望以上內(nèi)容可以對(duì)大家有一定的幫助,使各位可以學(xué)到更多知識(shí),如果覺(jué)得文章不錯(cuò),請(qǐng)把它分享出去讓更多的人看到。