GoldenGate作為Oracle公司大力發(fā)展的核心戰(zhàn)略產(chǎn)品,在高可用、容災(zāi)、
數(shù)據(jù)抽取轉(zhuǎn)換方面扮演著越來(lái)越重要的作用。goldengate可以做什么,說(shuō)起來(lái)真的太多了,
高可用、容災(zāi)、Real-Time數(shù)據(jù)同步,如果你做過(guò)BI,一定知道ETL的工具(datastage、infomatica等),
goldengate也可以做,而且,還有兩個(gè)很關(guān)鍵的因素,第一,goldengate支持異構(gòu)的數(shù)據(jù)庫(kù),幾乎囊括
了所有的主流數(shù)據(jù)庫(kù),到底哪些呢,
去這里看看吧http://www.oracle.com/technetwork/middleware/goldengate/downloads/index.html,
主流的DB2、MySQL、sql server、sysbase都支持。
第二,是oracle大力發(fā)展的核心戰(zhàn)略產(chǎn)品。
網(wǎng)站建設(shè)哪家好,找創(chuàng)新互聯(lián)!專注于網(wǎng)頁(yè)設(shè)計(jì)、網(wǎng)站建設(shè)、微信開發(fā)、微信小程序定制開發(fā)、集團(tuán)企業(yè)網(wǎng)站建設(shè)等服務(wù)項(xiàng)目。為回饋新老客戶創(chuàng)新互聯(lián)還提供了訥河免費(fèi)建站歡迎大家使用!
goldengate同步在源端捕獲日志,extract日志信息發(fā)送到目標(biāo)端,由目標(biāo)端的replicat日志應(yīng)用進(jìn)程
應(yīng)用事務(wù),達(dá)到數(shù)據(jù)同步。同時(shí),源端的extract抽取進(jìn)程也可以選擇性配置抽取的trail磁盤文件,
目標(biāo)端的replicat應(yīng)用進(jìn)程也可以選擇性配置抽取到的日志的trail文件,防止日志丟失。
類似售前性質(zhì)的話不多說(shuō),
本文精確的介紹了在64位linux環(huán)境下oracle對(duì)oracle的goldengate安裝配置(不區(qū)分10g、11g),
本文同時(shí)配置了batch模式下的數(shù)據(jù)初始化過(guò)程和online模式的實(shí)時(shí)數(shù)據(jù)同步過(guò)程,按照本文安裝配置
能讓你快速的構(gòu)建goldengate環(huán)境,不會(huì)有任何問(wèn)題。
1、環(huán)境介紹及準(zhǔn)備:
兩臺(tái)裝有oracle 10g軟件的centos(本文安裝配置同樣適用于oracle 11g).
源端: IP:192.168.1.201 ORACLE_SID=ggsource
目標(biāo)端: IP:192.168.1.202 ORACLE_SID=ggtarget
1.配置環(huán)境變量:
source:
ORACLE_SID=ggsource
ORACLE_BASE=/u01
ORACLE_HOME=$ORACLE_BASE/oracle
PATH=$ORACLE_HOME/bin:$ORACLE_BASE/gg:$PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORACLE_BASE/gg11:$LD_LIBRARY_PATH
export ORACLE_BASE ORACLE_HOME ORACLE_SID PATH LD_LIBRARY_PATH
target:
ORACLE_SID=ggtarget
ORACLE_BASE=/u01
ORACLE_HOME=$ORACLE_BASE/oracle
PATH=$ORACLE_HOME/bin:$ORACLE_BASE/gg:$PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORACLE_BASE/gg11:$LD_LIBRARY_PATH
export ORACLE_BASE ORACLE_HOME ORACLE_SID PATH LD_LIBRARY_PATH
其他的沒(méi)什么,搜索路勁加入了goldengate的安裝目錄$ORACLE_BASE/gg和LD_LIBRARY_PATH,
(后面在安裝goldengate時(shí),我的goldengate產(chǎn)品解壓安裝在$ORACLE_BASE/gg目錄下)
LD_LIBRARY_PATH一定要配置,否則在create subdirs會(huì)報(bào)如下錯(cuò)誤:
[oracle@mycentos02 ~]$ ggsci
ggsci: error while loading shared libraries: libnnz10.so: cannot open shared object file: No such file or directory
2.goldengate下載及安裝(需要同時(shí)在source機(jī)器和target機(jī)器安裝):
哪里下載?www.oracle.com->middleware->goldengate
怎么安裝?goldengate的安裝簡(jiǎn)單到嚇人,解壓就是安裝。將下載到的64位gg for oracle 10g,解壓到/u01/gg目錄下,
/u01/gg及為我的goldengate安裝目錄。解壓之后,cd到/u01/gg目錄下,運(yùn)行g(shù)gsci,在ggsci下面ggsci>create subdirs,
創(chuàng)建goldengate對(duì)應(yīng)的目錄,就算安裝完成。
GGSCI (mycentos) 1> create subdirs
Creating subdirectories under current directory /u01/gg
Parameter files /u01/gg/dirprm: already exists
Report files /u01/gg/dirrpt: created
Checkpoint files /u01/gg/dirchk: created
Process status files /u01/gg/dirpcs: created
SQL script files /u01/gg/dirsql: created
Database definitions files /u01/gg/dirdef: created
Extract data files /u01/gg/dirdat: created
Temporary files /u01/gg/dirtmp: created
Stdout files /u01/gg/dirout: created
GGSCI (mycentos) 2>
我們看到,在gg的安裝目錄下面建立了一些目錄,這些目錄是goldengate運(yùn)行過(guò)程中保存相關(guān)信息的目錄。譬如,
/u01/gg/dirrpt是report也就是goldengate的報(bào)表目錄, /u01/gg/dirprm是parameter目錄,也就是goldengate
進(jìn)程的參數(shù)配置目錄等等,其他的不再一一論述。
至此,goldengate安裝完成。
3.在source端增加supplemental log支持:
SQL> alter database add supplemental log data;
Database altered.
SQL> alter system switch logfile;
System altered.
SQL> select supplemental_log_data_min from v$database;
SUPPLEME
--------
YES
SQL>
4.在source端和target端同時(shí)創(chuàng)建gguser及其對(duì)應(yīng)的表空間:
SQL> create user gguser identified by oracle default tablespace users quota unlimited on users;
User created.
SQL> grant dba to gguser;
Grant succeeded.
SQL>
5.在source端和target端數(shù)據(jù)庫(kù)的gguser用戶下創(chuàng)建測(cè)試表,并在source端插入初始數(shù)據(jù):
source端:
SQL> create table ggtable(id number,name varchar2(50));
Table created.
SQL>
SQL> insert into ggtable values(1,'hunter01');
1 row created.
SQL> insert into ggtable values(2,'hunter02');
1 row created.
SQL> insert into ggtable values(3,'hunter03');
1 row created.
SQL> commit;
Commit complete.
SQL>
target端:
SQL> create table ggtable(id number,name varchar2(50));
Table created.
SQL>
6.在source和target端配置mgr進(jìn)程,并啟動(dòng):
manager是goldengate的管理進(jìn)程,在goldengate運(yùn)行時(shí)必須啟動(dòng),所以source和target兩端都要配置。
GGSCI (mycentos) 2> edit params mgr
GGSCI (mycentos) 3> start mgr
Manager started.
GGSCI (mycentos) 4> info mgr
Manager is running (IP port mycentos.7809).
GGSCI (mycentos) 5>
7.配置goldengate要同步的表:
GGSCI (mycentos) 1> dblogin userid gguser,password oracle
Successfully logged into database.
GGSCI (mycentos) 2> add trandata gguser.ggtable
2014-07-20 15:37:47 WARNING OGG-00869 No unique key is defined for table 'GGTABLE'. All viable columns will be used to represent the key, but may not guarantee uniqueness. KEYCOLS may be used to define the key.
Logging of supplemental redo data enabled for table GGUSER.GGTABLE.
GGSCI (mycentos) 3> info trandata gguser.*
Logging of supplemental redo log data is enabled for table GGUSER.GGTABLE.
Columns supplementally logged for table GGUSER.GGTABLE: ID, NAME.
GGSCI (mycentos) 4>
這里有一個(gè)表沒(méi)有unique key的warning,我們忽略,你也可以給表建立unique key.
8.完成initial load:
goldengate有batch和online兩種運(yùn)行模式,在online模式同步source端的數(shù)據(jù)之前,需要
初始化target端的數(shù)據(jù),當(dāng)然,如果在goldengate啟動(dòng)運(yùn)行同步之前你的源和
目標(biāo)端的數(shù)據(jù)一致,此步驟可以省略。另外,就算你要在goldengate運(yùn)行之前初始化兩端的數(shù)據(jù),
也可以用其他任何的數(shù)據(jù)復(fù)制和同步手段,例如exp/expdp等等。這里,是介紹goldengate的initial
load過(guò)程,并且我這里用的是direct load模式,其他模式我不詳細(xì)列舉,有興趣查閱goldengate的相關(guān)
文檔:
source端:
配置extract進(jìn)程并配置該進(jìn)程參數(shù):
GGSCI (mycentos) 2> add extract einikk,sourceistable
EXTRACT added.
GGSCI (mycentos) 3>
GGSCI (mycentos) 3> edit params einikk
參數(shù)文件:
EXTRACT EINIKK
USERID gguser, PASSWORD "oracle"
RMTHOST 192.168.1.202, MGRPORT 7809
RMTTASK REPLICAT, GROUP RINIKK
TABLE gguser.ggtable;
其中einikk為抽取的組名稱,sourceistable表示是一個(gè)initial load的task.
目標(biāo)端:
配置replicat進(jìn)程rinikk,并編輯該進(jìn)程的參數(shù):
add replicat rinikk,specialrun
edit params rinikk
參數(shù)文件:
REPLICAT RINIKK
ASSUMETARGETDEFS
USERID gguser, PASSWORD oracle
DISCARDFILE ./dirrpt/RINIKK.dsc, PURGE
MAP gguser.*, TARGET gguser.*;
開始initial load:
啟動(dòng)抽取進(jìn)程einikk,及進(jìn)行了initial load的過(guò)程:
GGSCI (mycentos) 5> start extract einikk
Sending START request to MANAGER ...
EXTRACT EINIKK starting
我們view report einikk查看einikk的報(bào)告,可以看到已經(jīng)完成了源端數(shù)據(jù)的初始化,
也就是在目標(biāo)端插入了初始的三條數(shù)據(jù):
Processing table GGUSER.GGTABLE
***********************************************************************
* ** Run Time Statistics ** *
***********************************************************************
Report at 2014-07-20 17:01:13 (activity since 2014-07-20 17:01:03)
Output to RINIKK:
From Table GGUSER.GGTABLE:
# inserts: 3
# updates: 0
# deletes: 0
# discards: 0
至此,initial load過(guò)程完成。
9.配置online模式的數(shù)據(jù)同步:
此過(guò)程也就是配置源端的extract進(jìn)程和目標(biāo)端的replicat進(jìn)程.
在源端配置抽取進(jìn)程eorakk,并配置該進(jìn)程參數(shù):
GGSCI (mycentos) 7> add extract eorakk,tranlog,begin now,threads 1
EXTRACT added.
GGSCI (mycentos) 8> info extract *
EXTRACT EORAKK Initialized 2014-07-20 17:09 Status STOPPED
Checkpoint Lag 00:00:00 (updated 00:00:13 ago)
Log Read Checkpoint Oracle Redo Logs
2014-07-20 17:09:47 Thread 1, Seqno 0, RBA 0
SCN 0.0 (0)
GGSCI (mycentos) 9>
GGSCI (mycentos) 9> edit params eorakk
EXTRACT EORAKK
USERID system, PASSWORD oracle
RMTHOST 192.168.1.202, MGRPORT 7809
RMTTRAIL ./dirdat/kk
TABLE gguser.ggtable;
在source端增加目標(biāo)端的trail文件:
add rmttrail ./dirdat/kk,extract eorakk
GGSCI (mycentos) 10> add rmttrail ./dirdat/kk,extract eorakk
RMTTRAIL added.
GGSCI (mycentos) 11> info rmttrail *
Extract Trail: ./dirdat/kk
Extract: EORAKK
Seqno: 0
RBA: 0
File Size: 100M
GGSCI (mycentos) 12>
然后,啟動(dòng)該抽取進(jìn)程:
GGSCI (mycentos) 12> start extract eorakk
Sending START request to MANAGER ...
EXTRACT EORAKK starting
GGSCI (mycentos) 13> info extract eorakk
EXTRACT EORAKK Last Started 2014-07-20 17:18 Status RUNNING
Checkpoint Lag 00:00:00 (updated 00:00:09 ago)
Log Read Checkpoint Oracle Redo Logs
2014-07-20 17:18:49 Thread 1, Seqno 4, RBA 2489344
SCN 0.578967 (578967)
GGSCI (mycentos) 14>
在目標(biāo)端配置replicat進(jìn)程,并啟動(dòng):
首先,配置checkpoint表:
edit params ./GLOBALS
GGSCI (mycentos02) 1> edit params ./GLoBALS
CHECKPOINTTABLE system.ggchkptable
創(chuàng)建ggchkptable:
GGSCI (mycentos02) 1>dblogin userid system,password oracle
add checkpointtable
這樣我們就建立了checkpointtable:system.ggchkptable,我們登陸到目標(biāo)端的system用戶下可以
看到該表:
[oracle@mycentos02 ~]$ sqlplus system/oracle
SQL*Plus: Release 10.2.0.1.0 - Production on Sun Jul 20 05:27:58 2014
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
With the Partitioning, OLAP and Data Mining options
SQL> desc ggchkptable
Name Null? Type
----------------------------------------- -------- ----------------------------
GROUP_NAME NOT NULL VARCHAR2(8)
GROUP_KEY NOT NULL NUMBER(19)
SEQNO NUMBER(10)
RBA NOT NULL NUMBER(19)
AUDIT_TS VARCHAR2(29)
CREATE_TS NOT NULL DATE
LAST_UPDATE_TS NOT NULL DATE
CURRENT_DIR NOT NULL VARCHAR2(255)
LOG_CSN VARCHAR2(129)
LOG_XID VARCHAR2(129)
LOG_CMPLT_CSN VARCHAR2(129)
LOG_CMPLT_XIDS VARCHAR2(2000)
VERSION NUMBER(3)
SQL>
配置replicat進(jìn)程rorakk,并啟動(dòng):
GGSCI (mycentos02) 1> add replicat rorakk,exttrail ./dirdat/kk
REPLICAT added.
GGSCI (mycentos02) 2>
參數(shù):
edit params rorakk
REPLICAT RINIKK
ASSUMETARGETDEFS
USERID gguser, PASSWORD oracle
DISCARDFILE ./dirrpt/RINIKK.dsc, PURGE
MAP gguser.*, TARGET gguser.*;
啟動(dòng):
start replicat rorakk
這樣我們就在source端和target端完成了goldengate數(shù)據(jù)同步的進(jìn)程,之后我們?cè)?br />源端的gguser.ggtable表中插入數(shù)據(jù)時(shí),可以在目標(biāo)端的數(shù)據(jù)庫(kù)中看到能達(dá)到數(shù)據(jù)同步,
同步有多快呢,oracle公司說(shuō)sub-second。
補(bǔ)充說(shuō)明:
goldengate的命令可以在ggsci下面輸入help進(jìn)程查看,進(jìn)程有stop、running和ABENDED狀態(tài),
正常運(yùn)行時(shí)是running狀態(tài),我在配置過(guò)程中,給rorakk進(jìn)程的參數(shù)文件中
USERID gguser, PASSWORD oracle剛開始寫錯(cuò)了gguser的密碼,并且,start之后,進(jìn)程在ABENDED,當(dāng)然也就無(wú)法同步數(shù)據(jù),
通過(guò)查看該進(jìn)程的report,
view report rorakk:
2014-07-20 05:43:43 ERROR OGG-00664 OCI Error beginning session (status = 1017-ORA-01017: invalid username/password; logon denie
d).
發(fā)現(xiàn)了該問(wèn)題。
另外,在映射表這里,MAP gguser.*, TARGET gguser.*;我也忘記寫了TARGET,也可以在該進(jìn)程的report里面檢查到了錯(cuò)誤原因:
2014-07-20 06:02:43 ERROR OGG-00212 Invalid option for MAP: gguser.ggtable.
改正了上述兩處錯(cuò)誤之后,我再次start rorakk,數(shù)據(jù)立即同步過(guò)來(lái)了。
總之,遇到錯(cuò)誤,要學(xué)會(huì)查看相關(guān)日志和報(bào)告,所有的命令,都可以help來(lái)查看。