ORA-00600: internal error code, arguments: [504] [row cache objects] on oracle 9i

ORA-00600: internal error code, arguments: [504] [row cache objects] on oracle 9i
# db alert log

Completed checkpoint up to RBA [0x46fc4.2.10], SCN: 0x085c.b4762046
Mon Nov 21 16:42:53 2016
Errors in file /ora/hbprd/db/weejar/9.2.0/admin/ANBOB/udump/hbprd_ora_9500.trc:
ORA-00600: internal error code, arguments: [504], [0x9FA72D138], [32], [4], [row cache objects], [15], [2], [0x000000000]
Mon Nov 21 16:42:56 2016
Errors in file /ora/hbprd/db/weejar/9.2.0/admin/ANBOB/udump/hbprd_ora_9500.trc:
ORA-00600: internal error code, arguments: [504], [0x9FA72D138], [32], [4], [row cache objects], [15], [2], [0x000000000]
Mon Nov 21 16:42:57 2016
Errors in file /ora/hbprd/db/weejar/9.2.0/admin/ANBOB/udump/hbprd_ora_9500.trc:
ORA-00600: internal error code, arguments: [504], [0x9FA72D138], [32], [4], [row cache objects], [15], [2], [0x000000000]
Mon Nov 21 16:42:57 2016
Errors in file /ora/hbprd/db/weejar/9.2.0/admin/ANBOB/udump/hbprd_ora_9500.trc:
ORA-00600: internal error code, arguments: [504], [0x9FA72D138], [32], [4], [row cache objects], [15], [2], [0x000000000]
Mon Nov 21 16:42:57 2016
Errors in file /ora/hbprd/db/weejar/9.2.0/admin/ANBOB/udump/hbprd_ora_9500.trc:
ORA-00600: internal error code, arguments: [504], [0x9FA72D138], [32], [4], [row cache objects], [15], [2], [0x000000000]
Mon Nov 21 16:42:57 2016
Errors in file /ora/hbprd/db/weejar/9.2.0/admin/ANBOB/udump/hbprd_ora_9500.trc:
ORA-07445: exception encountered: core dump [0000000101DEEFC4] [SIGSEGV] [Address not mapped to object] [0x1560000021F0001] [] []
ORA-00602: internal programming exception
ORA-00600: internal error code, arguments: [504], [0x9FA72D138], [32], [4], [row cache objects], [15], [2], [0x000000000]
Mon Nov 21 16:42:58 2016
Errors in file /ora/hbprd/db/weejar/9.2.0/admin/ANBOB/udump/hbprd_ora_9500.trc:
ORA-07445: exception encountered: core dump [0000000101DEEFC4] [SIGSEGV] [Address not mapped to object] [0x1560000021F0001] [] []
ORA-00602: internal programming exception
ORA-07445: exception encountered: core dump [0000000101DEEFC4] [SIGSEGV] [Address not mapped to object] [0x1560000021F0001] [] []
ORA-00602: internal programming exception
ORA-00600: internal error code, arguments: [504], [0x9FA72D138], [32], [4], [row cache objects], [15], [2], [0x000000000]
Mon Nov 21 16:42:58 2016
Errors in file /ora/hbprd/db/weejar/9.2.0/admin/ANBOB/udump/hbprd_ora_9500.trc:
ORA-07445: exception encountered: core dump [0000000101DEEFC4] [SIGSEGV] [Address not mapped to object] [0x1560000021F0001] [] []
ORA-00602: internal programming exception
ORA-07445: exception encountered: core dump [0000000101DEEFC4] [SIGSEGV] [Address not mapped to object] [0x1560000021F0001] [] []
ORA-00602: internal programming exception
ORA-07445: exception encountered: core dump [0000000101DEEFC4] [SIGSEGV] [Address not mapped to object] [0x1560000021F0001] [] []
ORA-00602: internal programming exception
ORA-00600: internal error code, arguments: [504], [0x9FA72D138], [32], [4], [row cache objects], [15], [2], [0x000000000]
Mon Nov 21 16:51:41 2016
复制

# trace file hbprd_ora_9500.trc

$ cat /ora/hbprd/db/weejar/9.2.0/admin/ANBOB/udump/hbprd_ora_9500.trc
Dump file /ora/hbprd/db/weejar/9.2.0/admin/ANBOB/udump/hbprd_ora_9500.trc
Oracle9i Enterprise Edition Release 9.2.0.6.0 - 64bit Production
With the Partitioning, OLAP and Oracle Data Mining options
JServer Release 9.2.0.6.0 - Production
ORACLE_HOME = /ora/hbprd/db/weejar/9.2.0
System name:    SunOS
Node name:      misdb
Release:        5.10
Version:        Generic_148888-01
Machine:        sun4v
Instance name: HBPRD
Redo thread mounted by this instance: 1
Oracle process number: 314
Unix process pid: 9500, image: oracle@misdb (TNS V1-V3)
*** SESSION ID:(261.51868) 2016-11-21 16:42:53.800
*** 2016-11-21 16:42:53.800
ksedmp: internal or fatal error
ORA-00600: internal error code, arguments: [504], [0x9FA72D138], [32], [4], [row cache objects], [15], [2], [0x000000000]
Current SQL statement for this session:
select * from v$sql_plan where address='0000000A45623BF8'
----- Call Stack Trace -----
calling              call     entry                 
location             type     point                 
-------------------- -------- -------------------- -
ksedmp()+328         CALL     ksedst()             
kgeriv()+208         PTR_CALL 0000000000000000     
kgesiv()+108         CALL     kgeriv()             
ksesic7()+92         CALL     kgesiv()             
kslgetl()+1096       CALL     ksesic7()            
kqrpre1()+372        CALL     kslgetl()            
kqrpre()+40          CALL     kqrpre1()            
npigdn0()+804        CALL     kqrpre()             
npicon0()+380        CALL     npigdn0()            
kpndbcon()+1372      CALL     npicon0()            
OCIKDBLinkConn()+52  CALL     kpndbcon()           
OCIKGetConn()+108    CALL     OCIKDBLinkConn()       # DBLINK
msqpfn()+464         CALL     OCIKGetConn()        
msqfun()+1408        CALL     msqpfn()             
k2copnws()+520       PTR_CALL 0000000000000000     
msqopnws()+3668      CALL     k2copnws()           
msqfun()+7172        CALL     msqopnws()           
k2copnws()+520       PTR_CALL 0000000000000000     
msqopnws()+3668      CALL     k2copnws()           
msqbin()+1952        CALL     msqopnws()           
k2copnws()+520       PTR_CALL 0000000000000000     
msqant()+1788        CALL     k2copnws()           
msq_logical()+640    CALL     msqant()             
xplPredToText()+560  CALL     msq_logical()        
xplGetPredicates()+  CALL     xplPredToText()      
252                                                
xplMakeRow()+524     CALL     xplGetPredicates()   
xplFetchRow()+172    PTR_CALL 0000000000000000     
kqlfgx()+596         CALL     xplFetchRow()        
kglic0()+976         PTR_CALL 0000000000000000     
kglic()+64           CALL     kglic0()             
kqlfxp()+504         CALL     kglic()              
qerfxFetch()+848     PTR_CALL 0000000000000000     
qerjoFetch()+308     PTR_CALL 0000000000000000     
qerjoFetch()+308     PTR_CALL 0000000000000000     
opifch2()+1724       PTR_CALL 0000000000000000     
opifch()+60          CALL     opifch2()            
opiodr()+1688        PTR_CALL 0000000000000000     
ttcpip()+1556        PTR_CALL 0000000000000000     
opitsk()+984         CALL     ttcpip()             
opiino()+1572        CALL     opitsk()             
opiodr()+1688        PTR_CALL 0000000000000000     
opidrv()+736         CALL     opiodr()             
sou2o()+16           CALL     opidrv()             
main()+184           CALL     sou2o()              
_start()+380         CALL     main() 
复制

My db env Oracle 9.2.0.6 on SunOS 5.10, during the problem occurs Database hang, and many sessions wait 'libaray cache lock' and after awhile sqlplus connect failed, Before the problem occurs ,We to a statspack, and explain plan for sql, but the sql did not use parallel and db_link, and select on v$sql_plan

Unpublished Bug:4434689
Queries on V$SQL_PLAN for SQL which used a database link
or PQ can fail with ORA-600 [504] errors. This can occur at high levels
of STATSPACK monitoring.
Versions Affected 9.2.0.6 & 9.2.0.7 fixed 9.2.0.8
Workaround
Use EXPLAIN PLAN against SQL obtained from V$SQL rather than using V$SQL_PLAN itself
or
One-off patch
or
upgrade 9.2.0.8

OTHER Note

unpublished Bug 4143776
Querying V$SQL_PLAN raises an ORA-600 [504] error on the "row cache objects" latch, as the session already held the library cache latch at an incompatible level.
Versions Affected 10.1.0.2 to 10.1.0.5 fixed in 10.2.0.1.
The fix for this is not backportable to previous releases.
The only solution is to upgrade to 10.2.0.1
There is a work-around for this problem if you stay in version 10.1 :
1. Set _cursor_plan_unparse_enabled=false:
a.- At session level:
* alter session set "_cursor_plan_unparse_enabled"=false
b.- At database level:
* In the init.ora or spfile.ora (check Note 137483.1) parameter file used to start up the database.
* Bounce the database in order to have the new parameter taken into account.
The one side effect is that the column PROJECTION and columns ending with "PREDICATES" are not populated. .

_cursor_plan_unparse_enabled
The undocument parameter introduce in Oracle 10.1.0
Parameter Name: _cursor_plan_unparse_enabled
Description: enables/disables using unparse to build projection/predicates
Type: BOOL Obsoleted: FALSE
Can ALTER SESSION: TRUE Can ALTER SYSTEM: IMMEDIATE