Troubleshooting Instance crash when modify db_cache_size, ora-600 [kmgs_pre_process_request_6]

昨天一套开发库在开发人员修改了db_cache_size后instance crash了， 10.2.0.1 single instance on RHEL linux,是因为触发了一个bug,这里简单记录一下.
# db alert
Mon Dec 22 22:00:58 2014
db_recovery_file_dest_size of 2048 MB is 0.38% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Mon Dec 22 22:01:07 2014
GATHER_STATS_JOB encountered errors. Check the trace file.
Mon Dec 22 22:01:07 2014
Errors in file /oracle/admin/anbobtest/bdump/anbobtest_j001_31269.trc:
ORA-20000: index “anbob_SHORT”.”SYS_MTABLE_0000260EA_IND_1″ or partition of such index is in unusable state
Mon Dec 22 22:40:42 2014
Errors in file /oracle/admin/anbobtest/bdump/anbobtest_mman_31205.trc:
ORA-00600: internal error code, arguments: [kmgs_pre_process_request_6], [6], [895], [0], [3], [0x459C1F3D8], [], []
Mon Dec 22 22:40:43 2014
Errors in file /oracle/admin/anbobtest/bdump/anbobtest_mman_31205.trc:
ORA-00600: internal error code, arguments: [kmgs_pre_process_request_6], [6], [895], [0], [3], [0x459C1F3D8], [], []
Mon Dec 22 22:40:43 2014
MMAN: terminating instance due to error 822
Instance terminated by MMAN, pid = 31205
$ vi /oracle/admin/anbobtest/bdump/anbobtest_mman_31205.trc

/oracle/admin/anbobtest/bdump/anbobtest_mman_31205.trc
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
With the Partitioning, OLAP and Data Mining options
ORACLE_HOME = /oracle/product/102
System name:    Linux
Node name:      OSS-TEST
Release:        2.6.32-358.el6.x86_64
Version:        #1 SMP Tue Jan 29 11:47:41 EST 2013
Machine:        x86_64
Instance name: anbobtest
Redo thread mounted by this instance: 1
Oracle process number: 4
Unix process pid: 31205, image: oracle@OSS-TEST (MMAN)
*** SERVICE NAME:(SYS$BACKGROUND) 2014-12-22 22:40:42.878
*** SESSION ID:(2203.1) 2014-12-22 22:40:42.878
*** 2014-12-22 22:40:42.878
ksedmp: internal or fatal error
ORA-00600: internal error code, arguments: [kmgs_pre_process_request_6], [6], [895], [0], [3], [0x459C1F3D8], [], []
----- Call Stack Trace -----
calling              call     entry                argument values in hex
location             type     point                (? means dubious value)
-------------------- -------- -------------------- ----------------------------
ksedst()+31          call     ksedst1()            000000000 ? 000000001 ?
                                                   000000000 ? 000000000 ?
                                                   000000000 ? 000000001 ?
ksedmp()+610         call     ksedst()             000000000 ? 000000001 ?
                                                   000000000 ? 000000000 ?
                                                   000000000 ? 000000001 ?
ksfdmp()+21          call     ksedmp()             000000003 ? 000000001 ?
                                                   000000000 ? 000000000 ?
                                                   000000000 ? 000000001 ?
kgerinv()+161        call     ksfdmp()             000000003 ? 000000001 ?
                                                   000000000 ? 000000000 ?
                                                   000000000 ? 000000001 ?
kgeasnmierr()+163    call     kgerinv()            006579D60 ? 7F22AFC90040 ?
                                                   000000000 ? 000000000 ?
                                                   000000001 ? 000000001 ?
kmgs_pre_process_re  call     kgeasnmierr()        006579D60 ? 7F22AFC90040 ?
quest()+2982                                       000000000 ? 000000000 ?
                                                   000000000 ? 000000006 ?
kmgsdrv()+1921       call     kmgs_pre_process_re  459C1F3D8 ? 7F22AFC90040 ?
                              quest()              000000000 ? 000000000 ?
                                                   000000000 ? 459C1F6A0 ?
ksbabs()+564         call     kmgsdrv()            7FFF102EF770 ? 000000010 ?
                                                   000000000 ? 000000000 ?
                                                   000000000 ? 459C1F6A0 ?
ksbrdp()+727         call     ksbabs()             7FFF102EF770 ? 000000010 ?
                                                   000000000 ? 000000000 ?
                                                   000000000 ? 459C1F6A0 ?
opirip()+616         call     ksbrdp()             7FFF102EF770 ? 000000010 ?
                                                   000000001 ? 0600117C8 ?
                                                   000000000 ? 459C1F6A0 ?
opidrv()+582         call     opirip()             000000032 ? 000000004 ?
                                                   7FFF102F0878 ? 0600117C8 ?
                                                   000000000 ? 459C1F6A0 ?
sou2o()+114          call     opidrv()             000000032 ? 000000004 ?
                                                   7FFF102F0878 ? 0600117C8 ?
                                                   000000000 ? 459C1F6A0 ?
Cannot find symbol
Cannot find symbol
Cannot find symbol
opimai_real()+317    call     sou2o()              7FFF102F0850 ? 000000032 ?
Cannot find symbol
Cannot find symbol
opimai_real()+317    call     sou2o()              7FFF102F0850 ? 000000032 ?
                                                   000000004 ? 7FFF102F0878 ?
                                                   000000000 ? 459C1F6A0 ?
main()+116           call     opimai_real()        000000003 ? 7FFF102F08E0 ?
                                                   000000004 ? 7FFF102F0878 ?
                                                   000000000 ? 459C1F6A0 ?
<0x36fb61ecdd>       call     main()               000000003 ? 7FFF102F08E0 ?
                                                   000000004 ? 7FFF102F0878 ?
                                                   000000000 ? 459C1F6A0 ?
复制

MOS note# 373802.1
MMON Terminates Instance With ORA-600 [kmgs_pre_process_request_6] or ORA-600 [kmgs_update_target_size_1] When Resizing Caches
ORA-00600: internal error code, arguments: [kmgs_update_target_size_1]
The call stack from the ORA-00600 trace file will be similar to:
… kmgs_pre_process_request <- kmgsdrv <- ksbabs <- ksbrdp <- opirip <- opidrv <- sou2o <- opimai_real <- main <- start or ... kmgs_update_target_size <- kmgs_update_parameter_ctx <- kmgs_update_param_manual_helper <- kmgs_update_param_manual
Cause
This is due to Bug:4433838 Similar issues have been filed as bugs which have also been determined to be duplicates of Bug:4433838 . The error occurs when the parameter SGA_TARGET is set to an exact multiple of 4Gb. For instance, if the SGA_TARGET value in HEXADECIMAL has 00000000 as the last 4 bytes then you can hit this problem. The bug indicates this happens on an exact multiple of 4G. Solution
1. Upgrade to 10.2.0.4 or higher.
OR
2 Apply the one-off Patch:4433838 depending on its availability for your patchset-level and platform
OR
3. Use the workaround: Set your SGA_TARGET parameter to any non-multiple values of 4 Gb. In other words, ensure that your SGA_TARGET value in HEXADECIMAL does not have 00000000 as the last 4 bytes.
我后来禁用了ASMM,直接把SGA_TARGET=0后，错误没有再出现