​Ogg  for bigdata实践问题处理分享

#在源端数据库操作，生成的表定义文件传输到OGGFOR BIGDATA端

visource.prm

defsfile./dirdef/source.def,purge

useridogg, PASSWORD ogg

tableusername.tablename;

tableusername.tablename;

………

defgenparamfile ogg/dirprm/source.prm

报错现象：

ERROR  OGG-03517  Conversion from character set zhs16gbk of source columnADDRESS to character set UTF-16 oftarget column ADDRESS fa

iledbecause the source column contains a character that is not availablein the target character set.

解决方案：

1）错误日志下方找到故障产生的队列文件和RBA号，为后天定位故障发生所在的事务

Reading./dirdat/yd000003234, current RBA 189904087, 131 records,m_file_seqno = 3234, m_file_rba = 189904087

2）通过logdump工具，检查这个SQL所在的事务，用Logdump:

./logdump

Logdump46 >open ./dirdat yd000003234 <--上一步获取

CurrentLogTrail is home/oracle/OGG_Target/./dirdat/yd000003234

Logdump47 >ghdr on

Logdump48 >detail data on

Logdump49 >usertoken on

Logdump50 >ggstoken detail

Logdump51 >pos 189904087 <--上一步获取

Logdump51 >n

Logdump211 >pos 189904087

Readingforward from RBA 189904087

Logdump212 >n

___________________________________________________________________

Hdr-Ind   :     E  (x45)     Partition  :     .  (x0c)  

UndoFlag  :     .  (x00)     BeforeAfter:     A  (x41)  

RecLength :   754  (x02f2)   IO Time    : 2019/02/03 07:56:41.139.230  

IOType    :    15  (x0f)     OrigNode   :   255  (xff)

TransInd  :     .  (x01)     FormatType :     R  (x52)

SyskeyLen :     0  (x00)     Incomplete :     .  (x00)

AuditRBA  :      97069       AuditPos   : 1030296448

Continued :     N  (x00)     RecCount   :     1  (x01)

2019/02/0307:56:41.139.230 FieldComp            Len   754 RBA 189904087

Name:DBTEST.USERINFO  (TDR Index: 13)

After Image:                                             Partition 12   G m  

0000000a 0000 0000 15e4 434f 19dd 0001 000a 0000 | ..........CO........  

00000000 0000 0001 0002 000a 0000 0006 a3aa a3aa | ....................  

c3b70003 0007 0000 0003 3030 3100 0400 0400 0031 | ..........001......1  

20000500 1f00 0000 1b33 3432 3532 33a3 aaa3 aaa3 |  ........342523.....  

aaa3aaa3 aaa3 aaa3 aaa3 aa37 3632 a3aa 0006 0014 | ...........762......  

00000010 a3aa a3aa a3aa a3aa a3aa a3aa a3aa a3aa | ....................  

00070015 0000 3230 3939 2d31 322d 3331 3a30 303a | ......2099-12-31:00:  

Column    0 (x0000), Len    10 (x000a)  

00000000 15e4 434f 19dd                          | ......CO..  

Column    1 (x0001), Len    10 (x000a)  

00000000 0000 0000 0001                          | ..........  

Column    2 (x0002), Len    10 (x000a)  

00000006 a3aa a3aa c3b7                          | ..........  

Column    3 (x0003), Len     7 (x0007)  

00000003 3030 31                                 | ....001  

Column    4 (x0004), Len     4 (x0004)  

说明：这里已经确定这个出问题的字段是DBTEST.USERINFO表。

3）通过配置文件中对应的表进行相关配置，重启进程即可

MAPDBTEST.USERINFO,TARGET DBTEST.USERINFO,COLCHARSET(PASSTHRU,ADDRESS);

报错现象：

OGG-03533 Conversion from character set zhs16gbk of source column ADDRESS tocharacter set UTF-8 of target column ADDRESS faile

dbecause the source column contains a character '84' at offset 24 thatis not available in the target character set.

解决方案：

在参数文件中加入以下参数，并重启进程

REPLACEBADCHARESCAPE

报错现象：

Theerror message:ERROR OGG-15051 Java or JNIexception:oracle.goldengate.util.GGException: Kafka Handler failed toformat and process operation: table=[DBTEST.USERINFO], oppos=00000054270047517956, tx pos=00000054270044951614, opts=2019-04-04 11:00

解决方案：

1）修改参数文件kafka.props

如果文件中有gg.handler.kafkahandler.format.pkUpdateHandling参数，则修改为delete-insert；如果为没有gg.handler.kafkahandler.format.pkUpdateHandling参数则新添加：

gg.handler.kafkahandler.format.pkUpdateHandling=delete-insert

2）保存退出后重启进程