GBase RTSync是南大通用的一款异构及同构数据库增量数据实时同步产品。为了保证数据同步的可用性,可以配置RTSync高可用模式。RTSync高可用依赖于Zookeeper服务。配置多套RTSync组件后,通过Zookeeper服务选举主节点后对外提供服务。当主节点服务异常退出后,通过Zookeeper服务会重新选举其他节点作为主来达到高可用的目的。
下面模拟一个项目来说明如何部署高可用服务
项目背景
项目需要将Oracle 11g中的增量数据同步到GBase 8s中,且该项目需要支持高可用服务。
部署环境
RTsync服务本身依赖于Zookeeper和kafka,为了保证Zookeeper和Kafka的高可用性,故准备了三台服务器,其ip为192.168.2.201,192.168.2.202,192.168.2.203。这三台机器用于部署三节点的zookeeper和kafka集群。
同时准备两台机器,其ip为192.168.2.204,192.168.2.205。这两台机器用于部署两套RTSync,用作高可用使用。
环境及工具安装
安装Zookeeper,将Zookeeper安装到相应的三台服务器上,可参考网上的教程;
安装Kafka,将Kafka安装到相应的三台服务器上,可参考网上的教程;
RTSync组件安装:
将RTSync分别复制到到204的/opt/目录下,目录为/opt/RTSync;205的/opt/目录下,目录为/opt/RTSync。
工具配置
首先配置RTSync使用的Zookeeper和Kafka,按照手册中的描述,修改config_kafka_ora.properties文件中的五个属性(其他属性采用默认即可):
bootstrap.servers=192.168. 2.201 :9092,192.168. 2.20 2:9092,192.168. 2.20 3:9092
zookeeper.connect= 192.168. 2.201 :2181,192.168. 2.20 2:2181,192.168. 2.20 3:2181
topic.name=test01
group.id=testgid
topic.replication.num=3
以上配置需要在204,205上同时修改。
然后配置RTSync的配置文件config_task.xml
针对204节点机器,修改配置为如下:
<manager ip = "192.168.2.20 4 " port = "9432" heartbeatPort = "9000" httpPort = "8080" isTableHotPatch = "true" />
<source ip = "192.168.2.20 4 " path = "/opt/RTSync" readParseAdapter = "adapter" user = "root" password = " ******** " queueSize = "10000" openMonitor = "true" monitorInterval = "300" rpcPort = "9191" sshPort = "22" dbObjToUpperCase = "false" isConvertSingleQuote = "true" queuePollTimeOut = "600" isEmptyStrPkEqualsNull = "true" isAllowInsertPkNull = "true" />
<target ip = "192.168.2.20 4 " path = "/opt/RTSync" writeDataAdapter = "adapter" user = "root" password = " ******** " errorishandle = " false " sendDataBySocket = " false " rpcPort = "9191" sshPort = "22" />
<mappings>
<source-target id = "sync1" >
<db>
<sourcedb
charset = "UTF8"
type = "ORACLE"
startLSN = "0"
fetchSize = "500"
oracleScnStep = "50000"
timestampWithFraction = "false"
maxRecordsPerRead = "200"
maxSizeOfPerRecord = "1024"
timeOut = "2"
driver = "oracle.jdbc.OracleDriver"
url = "jdbc:oracle:thin@192.168.2.108:1521/orcl"
user = " test "
password = " ******** "
catalog = " TEST "
parallel = "3" >
</sourcedb>
<targetdb
charset = "UTF8"
type = " GBASE8S "
commitSize = "1000"
queueSize = "2000"
user = " gbasedbt "
password = " ******** "
driver = "com. gbasedbt .jdbc.Driver"
catalog = "test"
timeOut = "2"
parrallel = " 3 "
url = "jdbc:gbase dbt-sqli ://192.168.2.105: 3000/sysmaster:gbasedbtserver=gbasetest " >
</targetdb>
<tables isInclude = "true" >
<table deleteMode = "NORMAL" sourceTableName = "T1" sourcePkColName = " COL1 " targetTableName = "t1" targetPkColName = " col1 " />
</tables>
</db>
</source-target>
</mappings>
针对205节点机器,修改配置为如下:
<manager ip = "192.168.2.20 5 " port = "9432" heartbeatPort = "9000" httpPort = "8080" isTableHotPatch = "true" />
<source ip = "192.168.2.20 5 " path = "/opt/RTSync" readParseAdapter = "adapter" user = "root" password = " ******** " queueSize = "10000" openMonitor = "true" monitorInterval = "300" rpcPort = "9191" sshPort = "22" dbObjToUpperCase = "false" isConvertSingleQuote = "true" queuePollTimeOut = "600" isEmptyStrPkEqualsNull = "true" isAllowInsertPkNull = "true" />
<target ip = "192.168.2.20 5 " path = "/opt/RTSync" writeDataAdapter = "adapter" user = "root" password = " ******** " errorishandle = " false " sendDataBySocket = " false " rpcPort = "9191" sshPort = "22" />
<mappings>
<source-target id = "sync1" >
<db>
<sourcedb
charset = "UTF8"
type = "ORACLE"
startLSN = "0"
fetchSize = "500"
oracleScnStep = "50000"
timestampWithFraction = "false"
maxRecordsPerRead = "200"
maxSizeOfPerRecord = "1024"
timeOut = "2"
driver = "oracle.jdbc.OracleDriver"
url = "jdbc:oracle:thin@192.168.2.108:1521/orcl"
user = " test "
password = " ******** "
catalog = " TEST "
parallel = "3" >
</sourcedb>
<targetdb
charset = "UTF8"
type = " GBASE8S "
commitSize = "1000"
queueSize = "2000"
user = " gbasedbt "
password = " ******** "
driver = "com. gbasedbt .jdbc.Driver"
catalog = "test"
timeOut = "2"
parrallel = " 3 "
url = "jdbc:gbase dbt-sqli ://192.168.2.105: 3000/sysmaster:gbasedbtserver=gbasetest " >
</targetdb>
<tables isInclude = "true" >
<table deleteMode = "NORMAL" sourceTableName = "T1" sourcePkColName = " COL1 " targetTableName = "t1" targetPkColName = " col1 " />
</tables>
</db>
</source-target>
</mappings>
注:为了进行测试,oracle建了个测试用户test,在其中建了张表T1,这里要注意的是Oracle中建表以后默认是大写表名,在这个配置文件中也要写成大写。
启动同步
启动Zookeeper和Kafka;
分别启动204及205节点上的RTSync,执行sh /opt/RTSync/RTSyncManagerServer.sh start命令;
因为优先启动的204节点,所以默认启动后204节点作为主对外提供服务。
测试同步
1)正常同步测试
同步工具已经启动好了,在oracle中插入一条数据,测试一下:
i nsert into t1 values( 1, ‘asdf’);
commit;
然后在GBase 8s执行查询语句:
select * from test: t1;
结果显示:
col1 1
col2 asdf
1 row(s) retrieved
好了,数据已经同步过来了。
2) 高可用切换测试
默认主节点是204节点(优先启动的是主节点),在204节点/opt/RTSync目录下执行./RTSyncManagerServer.sh stop,把当前主节点服务停掉。
然后在oracle中插入一条数据,测试一下:
i nsert into t1 values( 2 , ‘ def ’);
commit;
然后在GBase 8s执行查询语句:
select * from t1;
结果显示:
col1 1
col2 asdf
col1 2
col2 def
2 row(s) retrieved
好了,数据已经同步过来了。
总结
总体来说,高可用搭建比较简单,参考RTSync使用手册即可。需要注意下高可用开启的条件是修改config_task.xml的server节点属性,修改为如下即可
isHighAvailable=" true "
dataRecoveryMode=" zookeeper "
————————————————
版权声明:本文为CSDN博主「qq_39280087」的原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接及本声明。
原文链接:https://blog.csdn.net/qq_39280087/article/details/117152206