ogg学习系列--ORA-00353: log corruption due to GoldenGate extract process

某客户今天告诉我,一套核心库的备库的日志涨的很快，通过检查发现alert log不断的报错，如下所示：

Fri Dec 11 20:48:37 2015
Incomplete read from log member '/arch2/2_82548_864474703.dbf'. Trying next member.
Incomplete read from log member '/arch2/2_82548_864474703.dbf'. Trying next member.Archived Log entry 101817 added for thread 2 sequence 82548 ID 0xedd14d2a dest 1:
Incomplete read from log member '/arch2/2_82548_864474703.dbf'. Trying next member.
Errors in file /oracle/app/oracle/diag/rdbms/crmadg/crmdb2/trace/crmdb2_ora_15304.trc  (incident=711224):
ORA-00353: log corruption near block 900532 change 14410909668198 time 12/11/2015 20:42:40
ORA-00334: archived log: '/arch2/2_82548_864474703.dbf'
Errors in file /oracle/app/oracle/diag/rdbms/crmadg/crmdb2/trace/crmdb2_ora_15305.trc  (incident=711216):
ORA-00353: log corruption near block 900532 change 14410909668198 time 12/11/2015 20:42:40
ORA-00334: archived log: '/arch2/2_82548_864474703.dbf'
Fri Dec 11 20:48:37 2015
Errors in file /oracle/app/oracle/diag/rdbms/crmadg/crmdb2/trace/crmdb2_ora_13484.trc  (incident=711238):
ORA-00353: log corruption near block 900532 change 14410909668198 time 12/11/2015 20:42:40
ORA-00334: archived log: '/arch2/2_82548_864474703.dbf'
Fri Dec 11 20:49:13 2015
Sweep [inc][711238]: completed
复制

查看其中的trace可以看到如下的内容：

*** ACTION NAME:() 2015-12-10 21:58:10.036
 
kcrfrxini: Limitread is not enabled for standby db
kcrfrxini: Limitread is not enabled for standby db
kcrfrxini: Limitread is not enabled for standby db
....
复制

我google一把，发现居然毫无所获。。。从alert log最开始报错的时间点开始看，我怀疑跟最近部署的goldengate有关系。
后面我搜索关键字ORA-00353 log corruption  ogg extract 发现mos 论坛上有人遇到类似的问题，发帖者提到使用了如下的参数：

TranLogOptions DBLOGREADER
TranLogOptions BUFSIZE 4096000
TranLogOptions DBLOGREADERBUFSIZE 4096000
复制

该兄弟的环境为oracle 11.2.0.3 rac，ogg 版本为11.2.0.1.5；
最后据说通过将参数buffersize 、DBLOGREADERBUFSIZE 调小来解决该问题，最后貌似并没有提到解决了，只是暂时未出现。
考虑到客户这里是Oracle 11.2.0.4环境，虽然是ASM环境，然而归档日志都存放到本地文件系统上。

再次检查goldengate extract参数：

ogg_eda@xxx:/ogg_eda$ ./ggsci

Oracle GoldenGate Command Interpreter for Oracle
Version 11.2.1.0.27 19591627 OGGCORE_11.2.1.0.0OGGBP_PLATFORMS_141006.1156
HP/UX, IA64, 64bit (optimized), Oracle 11g on Oct  6 2014 13:37:58

Copyright (C) 1995, 2014, Oracle and/or its affiliates. All rights reserved.

GGSCI (xxx) 2> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                           
EXTRACT     RUNNING     DPEODS1     00:00:00      00:00:00    
EXTRACT     RUNNING     DPEODS2     00:00:00      00:00:00    
EXTRACT     RUNNING     DPEODS3     00:00:00      00:00:07    
EXTRACT     RUNNING     DPEODS4     00:00:00      00:00:08    
EXTRACT     RUNNING     EXTODS1     00:00:00      00:00:06    
EXTRACT     RUNNING     EXTODS2     00:00:00      00:00:00    
EXTRACT     RUNNING     EXTODS3     00:00:00      00:00:03    
EXTRACT     RUNNING     EXTODS4     00:00:00      00:00:03    


GGSCI (xxx) 3> view param EXTODS1

EXTRACT extods1
setenv (NLS_LANG = "AMERICAN_AMERICA.ZHS16GBK")
SETENV (ORACLE_HOME = "/oracle/app/oracle/product/11.2.0") 
SETENV (ORACLE_SID = "xxx2")
USERID ogg_eda, PASSWORD AACAAAAAAAAAAAIARCQIHGOIWCQJXITB ,ENCRYPTKEY default 
TRANLOGOPTIONS ALTARCHIVELOGDEST primary instance crmdb2 /arch2,ALTARCHIVELOGDEST primary instance crmdb1 /arch2
TRANLOGOPTIONS ARCHIVEDLOGONLY
TRANLOGOPTIONS ALTARCHIVEDLOGFORMAT {39ecd679003247f2ed728ad9c7ed019a369dd84d0731b449c26bf628d3c1a20b}t_{39ecd679003247f2ed728ad9c7ed019a369dd84d0731b449c26bf628d3c1a20b}s_{39ecd679003247f2ed728ad9c7ed019a369dd84d0731b449c26bf628d3c1a20b}r.dbf
TranlogOptions DBLOGREADER
REPORTCOUNT EVERY 1 MINUTES, RATE
DISCARDFILE ./dirrpt/extods1.dsc,APPEND,MEGABYTES 1024
DBOPTIONS  ALLOWUNUSEDCOLUMN
WARNLONGTRANS 2h,CHECKINTERVAL 3m
EXTTRAIL ./dirdat/ods/o1
FETCHOPTIONS NOUSESNAPSHOT
。。。。。
复制

由于是抽取active dataguard且仅仅抽取archivelog，因此完全没有必要加入TranlogOptions DBLOGREADER参数。
这里我怀疑跟该参数有关，通过屏蔽该参数后，发现不在报错。看来这是GoldengGate的未知bug呀！
在此记录一下，供大家参考！
参考文档：
GoldenGate Extract Archived Log Only (ALO) Mode Template Best Practices (文档 ID 1482439.1)Bug 13840711 - ORA-353 in Standby / Streams Data Capture or ORA-272 in PRIMARY: Redo log corruption by ASYNC redo shipping