我想将本地Oracle库中的表,用ogg同步到kafka中
现在的情况是,我所有的服务都已经起来了,并且日志也没有报错情况,源端和目标端的dirdat都有文件,但是在topic中查不到数据,麻烦帮我排除一下是什么原因呢,下面是我所用到的版本、服务以及kafka的配置文件,非常感谢!
源端(本地):Windows10,Oracle Database 11g,ogg_Windows_x64_Oracle_shiphome_21.3.0.0.1
目标端(虚机):Linux,OGG_BigData_Linux_x64_19.1.0.0.13, kafka_2.11
Linux节点服务情况:
Faded102节点服务
3344 JobHistoryServer
2992 DataNode
2737 QuorumPeerMain
2885 NameNode
3271 NodeManager
3386 RunJar
3660 Kafka
4094 Jps
4015
Faded103节点服务
2737 DataNode
2979 NodeManager
3364 Kafka
2869 ResourceManager
3789 Jps
2670 QuorumPeerMain
Faded104节点服务
2917 NodeManager
2727 DataNode
3547 Jps
2667 QuorumPeerMain
2797 SecondaryNameNode
3101 Kafka
Linux目标端Kafka配置文件:
[root@faded102 dirprm]# cat kafka.props
gg.handlerlist = kafkahandler
gg.handler.kafkahandler.type=kafka
gg.handler.kafkahandler.topicMappingTemplate=oggs0118
gg.handler.kafkahandler.KafkaProducerConfigFile=custom_kafka_producer.properties
gg.handler.kafkahandler.BlockingSend =false
gg.handler.kafkahandler.includeTokens=false
gg.handler.kafkahandler.mode=op
gg.handler.kafkahandler.format=json
gg.handler.kafkahandler.format.includePrimaryKeys=true
gg.handler.kafkahandler.format.insertOpKey=I
gg.handler.kafkahandler.format.updateOpKey=U
gg.handler.kafkahandler.format.deleteOpKey=D
gg.handler.kafkahandler.transactionsEnabled=false
goldengate.userexit.writers=javawriter
javawriter.stats.display=true
javawriter.stats.full=true
gg.log=log4j
gg.log.level=INFO
gg.report.time=30sec
gg.classpath=dirprm/:/opt/module/kafka_2.11-0.11.0.2/libs/*:/opt/module/ogg/:/opt/module/ogg/lib/*
javawriter.bootoptions=-Xmx512m -Xms32m -Djava.class.path=ggjava/ggjava.jar
[root@faded102 dirprm]# cat custom_kafka_producer.properties
bootstrap.servers=faded102:9092,faded103:9092,faded104:9092
acks=1
#compression.type=gzip
reconnect.backoff.ms=1000
value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
key.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
batch.size=16384
linger.ms=10000
ogg源端服务:
GGSCI (DESKTOP-VH529O0) 5> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING EXTKAFKA 00:00:00 00:00:00
EXTRACT RUNNING PUKAFKA 00:00:00 00:00:01
Linux目标端服务:
GGSCI (faded102) 4> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
REPLICAT RUNNING REKAFKA 00:00:00 00:00:00