您好请教一下,向生产环境的ASM文件系统的一个表空间添加一个数据文件,长时间不出结果,执行的命令:
ALTER TABLESPACE "TBS_IMDNH" ADD DataFile '+DATADG1/imddb/datafile/tbs_imdnh24.dbf' SIZE 34359148544 AUTOEXTEND ON;
alert日志内容如下:
Tue May 07 02:17:38 2019
Archived Log entry 51432 added for thread 2 sequence 24913 ID 0x3f36e0db dest 1:
Tue May 07 07:08:20 2019
Thread 2 advanced to log sequence 24915 (LGWR switch)
Current log# 3 seq# 24915 mem# 0: +DATADG1/imddb/onlinelog/group_3.265.839465291
Current log# 3 seq# 24915 mem# 1: +ADMINDG1/imddb/onlinelog/group_3.259.839465295
Tue May 07 07:08:39 2019
Archived Log entry 51433 added for thread 2 sequence 24914 ID 0x3f36e0db dest 1:
Tue May 07 10:53:59 2019
ALTER TABLESPACE "TBS_IMDNH" ADD DataFile '+DATADG1/imddb/datafile/tbs_imdnh24.dbf' SIZE 34359148544 AUTOEXTEND ON
Tue May 07 11:49:21 2019
Thread 2 advanced to log sequence 24916 (LGWR switch)
Current log# 4 seq# 24916 mem# 0: +DATADG1/imddb/onlinelog/group_4.266.839465297
Current log# 4 seq# 24916 mem# 1: +ADMINDG1/imddb/onlinelog/group_4.260.839465299
Tue May 07 11:49:39 2019
Archived Log entry 51435 added for thread 2 sequence 24915 ID 0x3f36e0db dest 1:
Tue May 07 12:59:50 2019
ORA-1013 signalled during: ALTER TABLESPACE "TBS_IMDNH" ADD DataFile '+DATADG1/imddb/datafile/tbs_imdnh24.dbf' SIZE 34359148544 AUTOEXTEND ON...
ALTER TABLESPACE "TBS_IMDNH" ADD DataFile '+DATADG1/imddb/datafile/tbs_imdnh24.dbf' SIZE 34359148544 AUTOEXTEND ON

您好
是想添加34g吗?
最好写成 ALTER TABLESPACE "TBS_IMDNH" ADD DataFile '+DATADG1/imddb/datafile/tbs_imdnh24.dbf' SIZE 34G AUTOEXTEND ON;


该表空间为smallfile表空间,按您的写法增加10G试过也是一样的效果


请检查一下做ALTER TABLESPACE的session的等待事件,添加文件大多等待在物理IO相关事件。


看日志加过多次,是不是手工中止过,之前看过asm日志及会话等待事件吗?
1、检查asm的alertlog
2、加小点的数据文件试试(如果你是8KB的blocksize,不是bigfile tablespace,最大数据文件最能是32GB)
3、做10046 trace 加文件的过程


感谢各位专家回复,查了下等待事件enq: TT - contention,其它insert语句也有相同等待事件,根据数据库版本和场景看跟Bug 8332021有几分相似。请专家看下后面除了打补丁或者重启还有没有好的办法能解决,直接杀掉相关等待事件对应的会话可以吗


最好是找到锁定源头,可以使用oradebug hanganalyze 3的方式,像这么长时间的tt锁,应该能够采集到。
如果问题已经消失,可以用以下sql尝试从历史性能数据中查找阻塞源头,第三个字段带有自动缩进,可以明显看到锁定链的关系:
select t.instance_number,
t.sample_time,
lpad('-', 2 * (level - 1), '-') ||
t.session_id,
t.BLOCKING_INST_ID,
t.blocking_session,
t.session_serial#,
t.sql_id,
t.event,
t.session_state,
level,
connect_by_isleaf,
connect_by_iscycle
from dba_hist_active_sess_history t
where sample_time between
to_date('20171020 19:00:00', 'yyyymmdd hh24:mi:ss') and
to_date('20171020 19:30:00', 'yyyymmdd hh24:mi:ss')
start with blocking_session is not null
and event like 'enq: TT - contention%'
connect by nocycle sample_time = prior sample_time
and session_id = prior blocking_session
and session_serial# = prior blocking_session_serial#
and instance_number = prior blocking_inst_id;


有2个add datafile语句吧,之前那个是不是没有结束?杀掉一个再试吧。


昨天执行了您提供的SQL并做了oradebug hanganalyze 3,结果如下:
Trace file /u01/app/oracle/diag/rdbms/imddb/imddb2/trace/imddb2_ora_41924.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/11.2.0/db_1
System name: Linux
Node name: mqdc-dbsr-ordb02
Release: 2.6.32-358.el6.x86_64
Version: #1 SMP Tue Jan 29 11:47:41 EST 2013
Machine: x86_64
Instance name: imddb2
Redo thread mounted by this instance: 2
Oracle process number: 227
Unix process pid: 41924, image: oracle@mqdc-dbsr-ordb02
*** 2019-04-30 09:22:54.197
*** CLIENT ID:() 2019-04-30 09:22:54.197
*** SERVICE NAME:() 2019-04-30 09:22:54.197
*** MODULE NAME:() 2019-04-30 09:22:54.197
*** ACTION NAME:() 2019-04-30 09:22:54.197
Breaking the connection before proto/dty negotiation, error raised 3113
Trace file /u01/app/oracle/diag/rdbms/imddb/imddb2/trace/imddb2_ora_41924.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/11.2.0/db_1
System name: Linux
Node name: mqdc-dbsr-ordb02
Release: 2.6.32-358.el6.x86_64
Version: #1 SMP Tue Jan 29 11:47:41 EST 2013
Machine: x86_64
Instance name: imddb2
Redo thread mounted by this instance: 2
Oracle process number: 134
Unix process pid: 41924, image: oracle@mqdc-dbsr-ordb02 (TNS V1-V3)
*** 2019-05-08 09:48:18.331
*** SESSION ID:(148.7612) 2019-05-08 09:48:18.331
*** CLIENT ID:() 2019-05-08 09:48:18.331
*** SERVICE NAME:(SYS$USERS) 2019-05-08 09:48:18.331
*** MODULE NAME:(sqlplus@mqdc-dbsr-ordb02 (TNS V1-V3)) 2019-05-08 09:48:18.331
*** ACTION NAME:() 2019-05-08 09:48:18.331
Processing Oradebug command 'hanganalyze 3'
*** 2019-05-08 09:48:20.215
===============================================================================
HANG ANALYSIS:
instances (db_name.oracle_sid): imddb.imddb2
oradebug_node_dump_level: 3
analysis initiated by oradebug
===============================================================================
Chains most likely to have caused the hang:
[a] Chain 1 Signature: 'enq: TT - contention'<='library cache lock'<='cursor: pin S wait on X'
Chain 1 Signature Hash: 0x717b6525
[b] Chain 2 Signature: 'enq: TT - contention'<='library cache lock'<='cursor: pin S wait on X'
Chain 2 Signature Hash: 0x717b6525
[c] Chain 3 Signature: 'enq: TT - contention'
Chain 3 Signature Hash: 0xae3bac3c
===============================================================================
Non-intersecting chains:
-------------------------------------------------------------------------------
Chain 1:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 30442
process id: 66, oracle@mqdc-dbsr-ordb02
session id: 51
session serial #: 23421
}
is waiting for 'cursor: pin S wait on X' with wait info:
{
p1: 'idn'=0x22f23065
p2: 'value'=0x3c100000000
p3: 'where'=0x500000000
time in wait: 7780 min 17 sec
timeout after: never
wait id: 89
blocking: 0 sessions
current sql: select HIN_ID,RBI_NUM,RBI_TYPE,HIN_FLOW,RBI_DIR,HIN_DATETIME from V_IMD_JC_HINGE_H_FLOW where HIN_ID = '117000000000000002'
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kksLockWait()+620<-kgxWait()+472<-kgxSharedExamine()+346<-kxsGetRuntimeLock()+217<-kkscsCheckCursor()+327<-kkscsSearchChildList()+1670<-kksSearchChildList()+86<-kksfbc()+10799<-kkspsc0()+1792<-kksParseCursor()+144<-opiosq0()+1920<-kpooprx()+295<-kpoal8()+779<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+1149<-opidrv()+570<-sou2o()+103<-opimai_real()+
wait history:
* time between current wait and wait #1: 0.000179 sec
1. event: 'SQL*Net message from client'
time waited: 0.007781 sec
wait id: 88 p1: 'driver id'=0x28444553
p2: '#bytes'=0x1
* time between wait #1 and #2: 0.000008 sec
2. event: 'SQL*Net message to client'
time waited: 0.000005 sec
wait id: 87 p1: 'driver id'=0x28444553
p2: '#bytes'=0x1
* time between wait #2 and #3: 0.000067 sec
3. event: 'SQL*Net message from client'
time waited: 52.657126 sec
wait id: 86 p1: 'driver id'=0x28444553
p2: '#bytes'=0x1
}
and is blocked by
=> Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 46244
process id: 104, oracle@mqdc-dbsr-ordb02
session id: 961
session serial #: 2841
}
which is waiting for 'library cache lock' with wait info:
{
p1: 'handle address'=0x78a2906a0
p2: 'lock address'=0x7ce5cd228
p3: '100*mode+namespace'=0x480300010002
time in wait: 7774 min 6 sec (last interval)
time in wait: 7838 min 7 sec (total)
timeout after: never
wait id: 1415
blocking: 2 sessions
current sql: select HIN_ID,RBI_NUM,RBI_TYPE,HIN_FLOW,RBI_DIR,HIN_DATETIME from V_IMD_JC_HINGE_H_FLOW where HIN_ID = '117000000000000002'
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-kqlmLock()+2428<-kqlmrac()+59<-kgllkal()+1679<-kglLock()+989<-kglget()+323<-kglgob()+187<-qcdlgbo()+509<-qcdlgob()+747<-qcsfgob()+241<-qcsprfro()+515<-qcsprfro_tree()+325<-qcsprfro_tree()+748<-qcsprfro_tree()+795<-qcsprfro_tree()+394<-qcspafq()+107<-qcspqbDescendents()+261<-qcspqb()+200<-qcsevw()+406<-kkmevw()+1887<-kkmfcbvw()+374<-kkmfcbl
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000005 sec
wait id: 1416 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'library cache lock'
time waited: 64 min 0 sec
wait id: 1415 p1: 'handle address'=0x78a2906a0
p2: 'lock address'=0x7ce5cd228
p3: '100*mode+namespace'=0x480300010002
* time between wait #2 and #3: 0.000171 sec
3. event: 'row cache lock'
time waited: 0.000327 sec
wait id: 1414 p1: 'cache id'=0x2
p2: 'mode'=0x0
p3: 'request'=0x3
}
and is blocked by
=> Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 47933
process id: 113, oracle@mqdc-dbsr-ordb02 (J001)
session id: 1178
session serial #: 11131
}
which is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 10652 min 16 sec (last interval)
time in wait: 10668 min 17 sec (total)
timeout after: never
wait id: 92
blocking: 3 sessions
current sql: <none>
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-kttget_alnq()+196<-ktsstrn_segment()+624<-kkbtis_trunc_idx_seg()+564<-kkbindl()+1040<-kkbind()+173<-kkbtrn()+1890<-opiexe()+17971<-opiosq0()+3942<-opipls()+11043<-opiodr()+1149<-__PGOSF141_rpidrus()+211<-skgmstack()+148<-rpiswu2()+617<-rpidrv()+1347<-psddr0()+464<-psdnal()+462<-pevm_EXIM()
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000004 sec
wait id: 93 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 92 p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000261 sec
3. event: 'db file sequential read'
time waited: 0.009189 sec
wait id: 91 p1: 'file#'=0x6
p2: 'block#'=0xb42
p3: 'blocks'=0x1
}
Chain 1 Signature: 'enq: TT - contention'<='library cache lock'<='cursor: pin S wait on X'
Chain 1 Signature Hash: 0x717b6525
-------------------------------------------------------------------------------
===============================================================================
Intersecting chains:
-------------------------------------------------------------------------------
Chain 2:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 17538
process id: 126, oracle@mqdc-dbsr-ordb02
session id: 1490
session serial #: 32259
}
is waiting for 'cursor: pin S wait on X' with wait info:
{
p1: 'idn'=0x22f23065
p2: 'value'=0x3c100000000
p3: 'where'=0x500000000
time in wait: 7821 min 52 sec
timeout after: never
wait id: 253
blocking: 0 sessions
current sql: select HIN_ID,RBI_NUM,RBI_TYPE,HIN_FLOW,RBI_DIR,HIN_DATETIME from V_IMD_JC_HINGE_H_FLOW where HIN_ID = '117000000000000002'
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kksLockWait()+620<-kgxWait()+472<-kgxSharedExamine()+346<-kxsGetRuntimeLock()+217<-kkscsCheckCursor()+327<-kkscsSearchChildList()+1670<-kksSearchChildList()+86<-kksfbc()+10799<-kkspsc0()+1792<-kksParseCursor()+144<-opiosq0()+1920<-kpooprx()+295<-kpoal8()+779<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+1149<-opidrv()+570<-sou2o()+103<-opimai_real()+
wait history:
* time between current wait and wait #1: 0.000143 sec
1. event: 'SQL*Net message from client'
time waited: 0.005899 sec
wait id: 252 p1: 'driver id'=0x28444553
p2: '#bytes'=0x1
* time between wait #1 and #2: 0.000010 sec
2. event: 'SQL*Net message to client'
time waited: 0.000006 sec
wait id: 251 p1: 'driver id'=0x28444553
p2: '#bytes'=0x1
* time between wait #2 and #3: 0.000057 sec
3. event: 'SQL*Net message from client'
time waited: 1 min 17 sec
wait id: 250 p1: 'driver id'=0x28444553
p2: '#bytes'=0x1
}
and is blocked by 'instance: 2, os id: 46244, session id: 961',
which is a member of 'Chain 1'.
Chain 2 Signature: 'enq: TT - contention'<='library cache lock'<='cursor: pin S wait on X'
Chain 2 Signature Hash: 0x717b6525
-------------------------------------------------------------------------------
===============================================================================
Sessions in an involuntary wait or not in a wait:
-------------------------------------------------------------------------------
Chain 3:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 75853
process id: 68, oracle@mqdc-dbsr-ordb02
session id: 100
session serial #: 1548
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 2964 min 31 sec
timeout after: never
wait id: 7008
blocking: 0 sessions
current sql: Insert into IMD_JC_LIVE_TRAFFIC (LT_ID,LT_datetime) values (lwltid.nextval, to_date('2019-05-06 08:32:00','yyyy-MM-dd hh24:mi:ss'))
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktsx_starttx()+92<-ktspnr_next()+554<-ktrsexec()+502<-ktspbmphwm()+1323<-ktspmvhwm()+47<-ktsp_bump_hwm()+212<-ktspgsp_main()+1508<-kdisnew()+276<-kdisnewle()+115<-kdisle()+3777<-kdiins0()+40810<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-__PGOSF635_qe
wait history:
* time between current wait and wait #1: 0.000163 sec
1. event: 'db file sequential read'
time waited: 0.012487 sec
wait id: 7007 p1: 'file#'=0xd
p2: 'block#'=0x284881
p3: 'blocks'=0x1
* time between wait #1 and #2: 0.000055 sec
2. event: 'gc cr grant 2-way'
time waited: 0.000327 sec
wait id: 7006 p1: ''=0xd
p2: ''=0x284881
p3: ''=0x8
* time between wait #2 and #3: 0.000037 sec
3. event: 'ges message buffer allocation'
time waited: 0.000003 sec
wait id: 7005 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
}
Chain 3 Signature: 'enq: TT - contention'
Chain 3 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 4:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 33405
process id: 133, oracle@mqdc-dbsr-ordb02 (W004)
session id: 123
session serial #: 10866
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 11306 min 18 sec
timeout after: never
wait id: 49
blocking: 0 sessions
current sql: <none>
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktte_ts_extend()+391<-ktsj_execute_task()+556<-ktsj_slave_main()+1252<-ksvrdp()+1737<-opirip()+735<-opidrv()+603<-sou2o()+103<-opimai_real()+266<-ssthrdmain()+214<-main()+201<-__libc_start_main()+253<-_start()+36
wait history:
* time between current wait and wait #1: 0.000636 sec
1. event: 'Space Manager: slave idle wait'
time waited: 3.940856 sec
wait id: 48 p1: 'Slave ID'=0x4
* time between wait #1 and #2: 0.000073 sec
2. event: 'Space Manager: slave idle wait'
time waited: 5.000933 sec
wait id: 47 p1: 'Slave ID'=0x4
* time between wait #2 and #3: 0.000023 sec
3. event: 'Space Manager: slave idle wait'
time waited: 5.000898 sec
wait id: 46 p1: 'Slave ID'=0x4
}
Chain 4 Signature: 'enq: TT - contention'
Chain 4 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 5:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 75895
process id: 85, oracle@mqdc-dbsr-ordb02
session id: 506
session serial #: 52669
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 2929 min 9 sec (last interval)
time in wait: 2945 min 10 sec (total)
timeout after: never
wait id: 34788
blocking: 0 sessions
current sql: Insert into IMD_JC_FLOW_msm_M (M_ID,ms_id,dir_id,M_DATETIME,CAR_MODEL,M_FLOW,m_avg_spped) values (smsmmid.nextval,17,2, to_date('2019-05-05 23:59:00','yyyy-MM-dd hh24:mi:ss'),8,1358,80.01)
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktsx_starttx()+92<-ktspnr_next()+554<-ktrsexec()+502<-ktspbmphwm()+1323<-ktspmvhwm()+47<-ktsp_bump_hwm()+212<-ktspgsp_main()+1508<-kdisnew()+276<-kdisnewle()+115<-kdisle()+3777<-kdiins0()+40810<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-__PGOSF635_qe
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000003 sec
wait id: 34789 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 34788 p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000131 sec
3. event: 'db file sequential read'
time waited: 0.019141 sec
wait id: 34787 p1: 'file#'=0x17
p2: 'block#'=0x18e280
p3: 'blocks'=0x1
}
Chain 5 Signature: 'enq: TT - contention'
Chain 5 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 6:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 21151
process id: 92, oracle@mqdc-dbsr-ordb02
session id: 674
session serial #: 37895
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540003
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 11336 min 21 sec (last interval)
time in wait: 11352 min 21 sec (total)
timeout after: never
wait id: 2960
blocking: 0 sessions
current sql: ALTER TABLESPACE "TBS_IMDNH" ADD DATAFILE '+DATADG1/imddb/datafile/tbs_imdnh24.dbf' REUSE AUTOEXTEND ON
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-atsdrv()+2046<-opiexe()+16187<-opiosq0()+3942<-kpooprx()+295<-kpoal8()+779<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+1149<-opidrv()+570<-sou2o()+103<-opimai_real()+133<-ssthrdmain()+214<-main()+201<-__libc_start_main()+253<-_start()+36
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000004 sec
wait id: 2961 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 2960 p1: 'name|mode'=0x54540003
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000021 sec
3. event: 'ges message buffer allocation'
time waited: 0.000002 sec
wait id: 2959 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
}
Chain 6 Signature: 'enq: TT - contention'
Chain 6 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 7:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 8112
process id: 93, oracle@mqdc-dbsr-ordb02
session id: 699
session serial #: 44228
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 11352 min 13 sec
timeout after: never
wait id: 22169854
blocking: 0 sessions
current sql: insert into IMD_JC_FLOW_HINGE_B_M1_XT(REC_ID,BA_CODE,HIN_CODE,DIR_ID,IO_ID,CAR_MODEL,LANE,UPDATE_TIME,START_TIME,END_TIME,FLOW_MAC,AVG_SPEED)
values
(SEQ_JCFLOW_H_B_XT.NEXTVAL,:1 ,:2 ,:3 ,:4 ,:5 ,:6 ,:7 ,:8 ,:9 ,:10 ,:11 )
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktsx_starttx()+92<-ktspnr_next()+554<-ktrsexec()+502<-ktspbmphwm()+1323<-ktspmvhwm()+47<-ktsp_bump_hwm()+212<-ktspgsp_main()+1508<-kdisnew()+276<-kdisnewle()+115<-kdisle()+3777<-kdiins0()+40810<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-__PGOSF635_qe
wait history:
* time between current wait and wait #1: 0.000132 sec
1. event: 'db file sequential read'
time waited: 0.000134 sec
wait id: 22169853 p1: 'file#'=0x4
p2: 'block#'=0x5889
p3: 'blocks'=0x1
* time between wait #1 and #2: 0.000152 sec
2. event: 'db file sequential read'
time waited: 0.000200 sec
wait id: 22169852 p1: 'file#'=0x4
p2: 'block#'=0x5888
p3: 'blocks'=0x1
* time between wait #2 and #3: 0.000438 sec
3. event: 'row cache lock'
time waited: 0.000061 sec
wait id: 22169851 p1: 'cache id'=0xd
p2: 'mode'=0x0
p3: 'request'=0x5
}
Chain 7 Signature: 'enq: TT - contention'
Chain 7 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 8:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 24943
process id: 94, oracle@mqdc-dbsr-ordb02 (W001)
session id: 724
session serial #: 26451
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 11320 min 19 sec (last interval)
time in wait: 11336 min 19 sec (total)
timeout after: never
wait id: 49
blocking: 0 sessions
current sql: <none>
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktte_ts_extend()+391<-ktsj_execute_task()+556<-ktsj_slave_main()+1252<-ksvrdp()+1737<-opirip()+735<-opidrv()+603<-sou2o()+103<-opimai_real()+266<-ssthrdmain()+214<-main()+201<-__libc_start_main()+253<-_start()+36
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000016 sec
wait id: 50 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 49 p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000787 sec
3. event: 'Space Manager: slave idle wait'
time waited: 3.322596 sec
wait id: 48 p1: 'Slave ID'=0x1
}
Chain 8 Signature: 'enq: TT - contention'
Chain 8 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 9:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 75907
process id: 101, oracle@mqdc-dbsr-ordb02
session id: 890
session serial #: 5
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 1283 min 29 sec (last interval)
time in wait: 1299 min 30 sec (total)
timeout after: never
wait id: 41115024
blocking: 0 sessions
current sql: Insert into IMD_JC_DC_PROD_STATUS_LOG (rec_ID,data_code,data_name,table_name,total_num,new_in_num,update_time,in_time) values (sijdpslid.nextval, 'PD0087','交调分钟统计数据','imd_jc_flow_in_jd_detail',14855500,4, to_date('2019-05-07 07:30:00','yyyy-MM-dd hh24:mi:ss'), to_date('2019-05-07 12:20:32','yyyy-MM-dd hh24:mi
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktsx_starttx()+92<-ktspnr_next()+554<-ktrsexec()+502<-ktspbmphwm()+1323<-ktspmvhwm()+47<-ktsp_bump_hwm()+212<-ktspgsp_main()+1508<-kdtgsp()+604<-kdtgsph()+2123<-kdtgrs()+273<-kdtInsRow()+1005<-__PGOSF635_qerltcNoKdtBufferedInsRowCBK()+293<-qerltcSingleRowLoad()+681<-qerltcFetch()+379<-inse
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000003 sec
wait id: 41115025 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 41115024 p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000149 sec
3. event: 'db file sequential read'
time waited: 0.008276 sec
wait id: 41115023 p1: 'file#'=0x20
p2: 'block#'=0x240687
p3: 'blocks'=0x1
}
Chain 9 Signature: 'enq: TT - contention'
Chain 9 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 10:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 27664
process id: 117, oracle@mqdc-dbsr-ordb02 (W002)
session id: 1274
session serial #: 47211
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 11326 min 19 sec
timeout after: never
wait id: 49
blocking: 0 sessions
current sql: <none>
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktte_ts_extend()+391<-ktsj_execute_task()+556<-ktsj_slave_main()+1252<-ksvrdp()+1737<-opirip()+735<-opidrv()+603<-sou2o()+103<-opimai_real()+266<-ssthrdmain()+214<-main()+201<-__libc_start_main()+253<-_start()+36
wait history:
* time between current wait and wait #1: 0.000662 sec
1. event: 'Space Manager: slave idle wait'
time waited: 3.545407 sec
wait id: 48 p1: 'Slave ID'=0x2
* time between wait #1 and #2: 0.000029 sec
2. event: 'Space Manager: slave idle wait'
time waited: 5.000943 sec
wait id: 47 p1: 'Slave ID'=0x2
* time between wait #2 and #3: 0.000030 sec
3. event: 'Space Manager: slave idle wait'
time waited: 5.000959 sec
wait id: 46 p1: 'Slave ID'=0x2
}
Chain 10 Signature: 'enq: TT - contention'
Chain 10 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 11:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 16709
process id: 56, oracle@mqdc-dbsr-ordb02 (W000)
session id: 1345
session serial #: 4111
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 11346 min 19 sec
timeout after: never
wait id: 365
blocking: 0 sessions
current sql: <none>
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktte_ts_extend()+391<-ktsj_execute_task()+556<-ktsj_slave_main()+1252<-ksvrdp()+1737<-opirip()+735<-opidrv()+603<-sou2o()+103<-opimai_real()+266<-ssthrdmain()+214<-main()+201<-__libc_start_main()+253<-_start()+36
wait history:
* time between current wait and wait #1: 0.000128 sec
1. event: 'Space Manager: slave idle wait'
time waited: 3.186509 sec
wait id: 364 p1: 'Slave ID'=0x0
* time between wait #1 and #2: 0.000033 sec
2. event: 'Space Manager: slave idle wait'
time waited: 5.001009 sec
wait id: 363 p1: 'Slave ID'=0x0
* time between wait #2 and #3: 0.000027 sec
3. event: 'Space Manager: slave idle wait'
time waited: 5.000890 sec
wait id: 362 p1: 'Slave ID'=0x0
}
Chain 11 Signature: 'enq: TT - contention'
Chain 11 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 12:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 30424
process id: 123, oracle@mqdc-dbsr-ordb02 (W003)
session id: 1417
session serial #: 51284
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 11300 min 18 sec (last interval)
time in wait: 11316 min 18 sec (total)
timeout after: never
wait id: 49
blocking: 0 sessions
current sql: <none>
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktte_ts_extend()+391<-ktsj_execute_task()+556<-ktsj_slave_main()+1252<-ksvrdp()+1737<-opirip()+735<-opidrv()+603<-sou2o()+103<-opimai_real()+266<-ssthrdmain()+214<-main()+201<-__libc_start_main()+253<-_start()+36
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000011 sec
wait id: 50 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 49 p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000692 sec
3. event: 'Space Manager: slave idle wait'
time waited: 3.778618 sec
wait id: 48 p1: 'Slave ID'=0x3
}
Chain 12 Signature: 'enq: TT - contention'
Chain 12 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 13:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 37894
process id: 124, oracle@mqdc-dbsr-ordb02
session id: 1443
session serial #: 3738
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540003
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 1232 min 24 sec (last interval)
time in wait: 1248 min 24 sec (total)
timeout after: never
wait id: 49
blocking: 0 sessions
current sql: ALTER TABLESPACE "TBS_IMDNH" ADD DataFile '+DATADG1/imddb/datafile/tbs_imdnh24.dbf' SIZE 34359148544 AUTOEXTEND ON
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-atsdrv()+2046<-opiexe()+16187<-opiosq0()+3942<-kpooprx()+295<-kpoal8()+779<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+1149<-opidrv()+570<-sou2o()+103<-opimai_real()+133<-ssthrdmain()+214<-main()+201<-__libc_start_main()+253<-_start()+36
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000003 sec
wait id: 50 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 49 p1: 'name|mode'=0x54540003
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000561 sec
3. event: 'library cache pin'
time waited: 0.000250 sec
wait id: 48 p1: 'handle address'=0x76a918a70
p2: 'pin address'=0x7ae2ec790
p3: '100*mode+namespace'=0x34a600030002
}
Chain 13 Signature: 'enq: TT - contention'
Chain 13 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 14:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 75849
process id: 62, oracle@mqdc-dbsr-ordb02
session id: 1492
session serial #: 11684
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 2623 min 46 sec (last interval)
time in wait: 2639 min 47 sec (total)
timeout after: never
wait id: 1158621
blocking: 0 sessions
current sql: Insert into IMD_JC_FLOW_SA_PARK (Sp_ID,SA_ID,sp_DATETIME) values (ssaspid.nextval,111000000000000006, to_date('2019-05-06 14:00:00','yyyy-MM-dd hh24:mi:ss'))
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktsx_starttx()+92<-ktspnr_next()+554<-ktrsexec()+502<-ktspbmphwm()+1323<-ktspmvhwm()+47<-ktsp_bump_hwm()+212<-ktspgsp_main()+1508<-kdtgsp()+604<-kdtgsph()+2123<-kdtgrs()+273<-kdtInsRow()+1005<-__PGOSF635_qerltcNoKdtBufferedInsRowCBK()+293<-qerltcSingleRowLoad()+681<-qerltcFetch()+379<-inse
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000004 sec
wait id: 1158622 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 1158621 p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000168 sec
3. event: 'db file sequential read'
time waited: 0.010550 sec
wait id: 1158620 p1: 'file#'=0x1d
p2: 'block#'=0x9fc81
p3: 'blocks'=0x1
}
Chain 14 Signature: 'enq: TT - contention'
Chain 14 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
===============================================================================
Extra information that will be dumped at higher levels:
[level 4] : 1 node dumps -- [LEAF] [LEAF_NW]
[level 5] : 15 node dumps -- [NO_WAIT] [INVOL_WT] [SINGLE_NODE] [NLEAF] [SINGLE_NODE_NW]
State of ALL nodes
([nodenum]/cnode/sid/sess_srno/session/ospid/state/[adjlist]):
[50]/2/51/23421/0x731041dd8/30442/NLEAF/[960]
[99]/2/100/1548/0x7d10545b8/75853/SINGLE_NODE/
[122]/2/123/10866/0x711081ed0/33405/SINGLE_NODE/
[505]/2/506/52669/0x7111218e0/75895/SINGLE_NODE/
[673]/2/674/37895/0x711167360/21151/SINGLE_NODE/
[698]/2/699/44228/0x7511750d0/8112/SINGLE_NODE/
[723]/2/724/26451/0x7411636e8/24943/SINGLE_NODE/
[889]/2/890/5/0x72121ea58/75907/SINGLE_NODE/
[960]/2/961/2841/0x7511e3570/46244/NLEAF/[1177]
[1177]/2/1178/11131/0x7112382e0/47933/LEAF/
[1273]/2/1274/47211/0x73123dc68/27664/SINGLE_NODE/
[1344]/2/1345/4111/0x701275368/16709/SINGLE_NODE/
[1416]/2/1417/51284/0x7d1278e68/30424/SINGLE_NODE/
[1442]/2/1443/3738/0x74128e5f8/37894/SINGLE_NODE/
[1489]/2/1490/32259/0x7412a2b08/17538/NLEAF/[960]
[1491]/2/1492/11684/0x7512bd040/75849/SINGLE_NODE/
*** 2019-05-08 09:48:20.233
===============================================================================
END OF HANG ANALYSIS
===============================================================================
*** 2019-05-08 09:48:20.235
===============================================================================
HANG ANALYSIS DUMPS:
oradebug_node_dump_level: 3
===============================================================================
State of LOCAL nodes
([nodenum]/cnode/sid/sess_srno/session/ospid/state/[adjlist]):
[50]/2/51/23421/0x731041dd8/30442/NLEAF/[960]
[99]/2/100/1548/0x7d10545b8/75853/SINGLE_NODE/
[122]/2/123/10866/0x711081ed0/33405/SINGLE_NODE/
[505]/2/506/52669/0x7111218e0/75895/SINGLE_NODE/
[673]/2/674/37895/0x711167360/21151/SINGLE_NODE/
[698]/2/699/44228/0x7511750d0/8112/SINGLE_NODE/
[723]/2/724/26451/0x7411636e8/24943/SINGLE_NODE/
[889]/2/890/5/0x72121ea58/75907/SINGLE_NODE/
[960]/2/961/2841/0x7511e3570/46244/NLEAF/[1177]
[1177]/2/1178/11131/0x7112382e0/47933/LEAF/
[1273]/2/1274/47211/0x73123dc68/27664/SINGLE_NODE/
[1344]/2/1345/4111/0x701275368/16709/SINGLE_NODE/
[1416]/2/1417/51284/0x7d1278e68/30424/SINGLE_NODE/
[1442]/2/1443/3738/0x74128e5f8/37894/SINGLE_NODE/
[1489]/2/1490/32259/0x7412a2b08/17538/NLEAF/[960]
[1491]/2/1492/11684/0x7512bd040/75849/SINGLE_NODE/
No processes qualify for dumping.
===============================================================================
HANG ANALYSIS DUMPS: END
===============================================================================
*** 2019-05-08 09:48:20.235
Oradebug command 'hanganalyze 3' console output:
Hang Analysis in /u01/app/oracle/diag/rdbms/imddb/imddb2/trace/imddb2_ora_41924.trc


根据昨天执行的上个hanganalyze结果我kill掉了1178那个会话,今天重新执行了一下oradebug hanganalyze 3
Trace file /u01/app/oracle/diag/rdbms/imddb/imddb2/trace/imddb2_ora_51152.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/11.2.0/db_1
System name: Linux
Node name: mqdc-dbsr-ordb02
Release: 2.6.32-358.el6.x86_64
Version: #1 SMP Tue Jan 29 11:47:41 EST 2013
Machine: x86_64
Instance name: imddb2
Redo thread mounted by this instance: 2
Oracle process number: 130
Unix process pid: 51152, image: oracle@mqdc-dbsr-ordb02 (TNS V1-V3)
*** 2019-05-09 10:48:33.457
*** SESSION ID:(50.12596) 2019-05-09 10:48:33.457
*** CLIENT ID:() 2019-05-09 10:48:33.457
*** SERVICE NAME:(SYS$USERS) 2019-05-09 10:48:33.457
*** MODULE NAME:(sqlplus@mqdc-dbsr-ordb02 (TNS V1-V3)) 2019-05-09 10:48:33.457
*** ACTION NAME:() 2019-05-09 10:48:33.457
*** TRACE FILE RECREATED AFTER BEING REMOVED ***
*** 2019-05-09 10:48:33.457
===============================================================================
HANG ANALYSIS:
instances (db_name.oracle_sid): imddb.imddb2
oradebug_node_dump_level: 3
analysis initiated by oradebug
===============================================================================
Chains most likely to have caused the hang:
[a] Chain 1 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 1 Signature Hash: 0x7aa23cc3
[b] Chain 2 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 2 Signature Hash: 0x7aa23cc3
[c] Chain 3 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 3 Signature Hash: 0x7aa23cc3
===============================================================================
Non-intersecting chains:
-------------------------------------------------------------------------------
Chain 1:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 55537
process id: 131, oracle@mqdc-dbsr-ordb02
session id: 75
session serial #: 6778
}
is waiting for 'enq: TX - index contention' with wait info:
{
p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
time in wait: 1274 min 29 sec
timeout after: never
wait id: 121
blocking: 0 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT,EVENT_LON_2000,EVENT_LAT_2000) VALUES ('00120190508000014
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktcwit1()+330<-ktbgtl0()+1250<-kdiins0()+3521<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleRowLoad()+366<-qerltcFetch()+379<-insexe()+1106<-opiexe()+4683<-kpoal8()+2288<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+11
wait history:
* time between current wait and wait #1: 0.000306 sec
1. event: 'gc current grant busy'
time waited: 0.000671 sec
wait id: 120 p1: ''=0x22
p2: ''=0x2ca829
p3: ''=0x2010001
* time between wait #1 and #2: 0.000032 sec
2. event: 'ges message buffer allocation'
time waited: 0.000009 sec
wait id: 119 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #2 and #3: 0.001968 sec
3. event: 'rdbms ipc reply'
time waited: 0.000103 sec
wait id: 118 p1: 'from_process'=0x25
p2: 'timeout'=0x7fffffff
}
and is blocked by
=> Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 45767
process id: 122, oracle@mqdc-dbsr-ordb02
session id: 1395
session serial #: 9988
}
which is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 1280 min 41 sec (last interval)
time in wait: 1296 min 42 sec (total)
timeout after: never
wait id: 227
blocking: 14 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT) VALUES ('001201905080000140205316',-1833740193,'平时交通
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktsx_starttx()+92<-ktspnr_next()+554<-ktrsexec()+502<-ktspbmphwm()+1323<-ktspmvhwm()+47<-ktsp_bump_hwm()+212<-ktspgsp_main()+1508<-kdisnew()+276<-kdisnewle()+115<-kdisle()+3777<-kdiins0()+40810<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleR
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000010 sec
wait id: 228 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 227 p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000442 sec
3. event: 'db file sequential read'
time waited: 0.000411 sec
wait id: 226 p1: 'file#'=0x4
p2: 'block#'=0xbac
p3: 'blocks'=0x1
}
Chain 1 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 1 Signature Hash: 0x7aa23cc3
-------------------------------------------------------------------------------
===============================================================================
Intersecting chains:
-------------------------------------------------------------------------------
Chain 2:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 72076
process id: 73, oracle@mqdc-dbsr-ordb02
session id: 218
session serial #: 52069
}
is waiting for 'enq: TX - index contention' with wait info:
{
p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
time in wait: 1230 min 15 sec
timeout after: never
wait id: 149
blocking: 0 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT,EVENT_LON_2000,EVENT_LAT_2000) VALUES ('00120190508000014
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktcwit1()+330<-ktbgtl0()+1250<-kdiins0()+3521<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleRowLoad()+366<-qerltcFetch()+379<-insexe()+1106<-opiexe()+4683<-kpoal8()+2288<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+11
wait history:
* time between current wait and wait #1: 0.000432 sec
1. event: 'gc current grant busy'
time waited: 0.000872 sec
wait id: 148 p1: ''=0x22
p2: ''=0x2ca85c
p3: ''=0x2010001
* time between wait #1 and #2: 0.000046 sec
2. event: 'ges message buffer allocation'
time waited: 0.000004 sec
wait id: 147 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #2 and #3: 0.001872 sec
3. event: 'SQL*Net message from client'
time waited: 0.005676 sec
wait id: 146 p1: 'driver id'=0x28444553
p2: '#bytes'=0x1
}
and is blocked by 'instance: 2, os id: 45767, session id: 1395',
which is a member of 'Chain 1'.
Chain 2 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 2 Signature Hash: 0x7aa23cc3
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 3:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 56679
process id: 139, oracle@mqdc-dbsr-ordb02
session id: 267
session serial #: 6810
}
is waiting for 'enq: TX - index contention' with wait info:
{
p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
time in wait: 1205 min 59 sec (last interval)
time in wait: 1269 min 59 sec (total)
timeout after: never
wait id: 268
blocking: 0 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT,EVENT_LON_2000,EVENT_LAT_2000) VALUES ('00120190508000014
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktcwit1()+330<-ktbgtl0()+1250<-kdiins0()+3521<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleRowLoad()+366<-qerltcFetch()+379<-insexe()+1106<-opiexe()+4683<-kpoal8()+2288<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+11
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000009 sec
wait id: 269 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TX - index contention'
time waited: 64 min 0 sec
wait id: 268 p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
* time between wait #2 and #3: 0.000325 sec
3. event: 'gc current grant busy'
time waited: 0.000582 sec
wait id: 267 p1: ''=0x22
p2: ''=0x2ca831
p3: ''=0x2010001
}
and is blocked by 'instance: 2, os id: 45767, session id: 1395',
which is a member of 'Chain 1'.
Chain 3 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 3 Signature Hash: 0x7aa23cc3
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 4:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 37926
process id: 96, oracle@mqdc-dbsr-ordb02
session id: 771
session serial #: 49073
}
is waiting for 'enq: TX - index contention' with wait info:
{
p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
time in wait: 1226 min 24 sec (last interval)
time in wait: 1290 min 25 sec (total)
timeout after: never
wait id: 1249
blocking: 0 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT,EVENT_LON_2000,EVENT_LAT_2000) VALUES ('00120190508000014
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktcwit1()+330<-ktbgtl0()+1250<-kdiins0()+3521<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleRowLoad()+366<-qerltcFetch()+379<-insexe()+1106<-opiexe()+4683<-kpoal8()+2288<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+11
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000011 sec
wait id: 1250 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TX - index contention'
time waited: 64 min 0 sec
wait id: 1249 p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
* time between wait #2 and #3: 0.000277 sec
3. event: 'gc current grant busy'
time waited: 0.000672 sec
wait id: 1248 p1: ''=0x22
p2: ''=0x2ca819
p3: ''=0x2010001
}
and is blocked by 'instance: 2, os id: 45767, session id: 1395',
which is a member of 'Chain 1'.
Chain 4 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 4 Signature Hash: 0x7aa23cc3
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 5:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 53065
process id: 103, oracle@mqdc-dbsr-ordb02
session id: 938
session serial #: 12418
}
is waiting for 'enq: TX - index contention' with wait info:
{
p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
time in wait: 1281 min 56 sec
timeout after: never
wait id: 106
blocking: 0 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT) VALUES ('001201905080000140205316',-1833740193,'平时交通
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktcwit1()+330<-ktbgtl0()+1250<-kdiins0()+3521<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleRowLoad()+366<-qerltcFetch()+379<-insexe()+1106<-opiexe()+4683<-kpoal8()+2288<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+11
wait history:
* time between current wait and wait #1: 0.001918 sec
1. event: 'rdbms ipc reply'
time waited: 0.000202 sec
wait id: 105 p1: 'from_process'=0x25
p2: 'timeout'=0x7fffffff
* time between wait #1 and #2: 0.000072 sec
2. event: 'rdbms ipc reply'
time waited: 0.000185 sec
wait id: 104 p1: 'from_process'=0x25
p2: 'timeout'=0x7fffffff
* time between wait #2 and #3: 0.000083 sec
3. event: 'rdbms ipc reply'
time waited: 0.000200 sec
wait id: 103 p1: 'from_process'=0x25
p2: 'timeout'=0x7fffffff
}
and is blocked by 'instance: 2, os id: 45767, session id: 1395',
which is a member of 'Chain 1'.
Chain 5 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 5 Signature Hash: 0x7aa23cc3
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 6:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 53482
process id: 109, oracle@mqdc-dbsr-ordb02
session id: 1084
session serial #: 49939
}
is waiting for 'enq: TX - index contention' with wait info:
{
p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
time in wait: 1280 min 49 sec
timeout after: never
wait id: 31
blocking: 0 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT) VALUES ('001201905080000140205316',-1833740193,'平时交通
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktcwit1()+330<-ktbgtl0()+1250<-kdiins0()+3521<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleRowLoad()+366<-qerltcFetch()+379<-insexe()+1106<-opiexe()+4683<-kpoal8()+2288<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+11
wait history:
* time between current wait and wait #1: 0.000484 sec
1. event: 'gc current grant busy'
time waited: 0.000889 sec
wait id: 30 p1: ''=0x22
p2: ''=0x2ca813
p3: ''=0x2010001
* time between wait #1 and #2: 0.000046 sec
2. event: 'ges message buffer allocation'
time waited: 0.000005 sec
wait id: 29 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #2 and #3: 0.001779 sec
3. event: 'rdbms ipc reply'
time waited: 0.000038 sec
wait id: 28 p1: 'from_process'=0x25
p2: 'timeout'=0x7fffffff
}
and is blocked by 'instance: 2, os id: 45767, session id: 1395',
which is a member of 'Chain 1'.
Chain 6 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 6 Signature Hash: 0x7aa23cc3
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 7:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 55134
process id: 111, oracle@mqdc-dbsr-ordb02
session id: 1130
session serial #: 12539
}
is waiting for 'enq: TX - index contention' with wait info:
{
p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
time in wait: 1276 min 35 sec
timeout after: never
wait id: 57
blocking: 0 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT,EVENT_LON_2000,EVENT_LAT_2000) VALUES ('00120190508000014
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktcwit1()+330<-ktbgtl0()+1250<-kdiins0()+3521<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleRowLoad()+366<-qerltcFetch()+379<-insexe()+1106<-opiexe()+4683<-kpoal8()+2288<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+11
wait history:
* time between current wait and wait #1: 0.000342 sec
1. event: 'gc current grant busy'
time waited: 0.000668 sec
wait id: 56 p1: ''=0x22
p2: ''=0x2ca815
p3: ''=0x2010001
* time between wait #1 and #2: 0.000021 sec
2. event: 'ges message buffer allocation'
time waited: 0.000012 sec
wait id: 55 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #2 and #3: 0.000148 sec
3. event: 'gc current grant busy'
time waited: 0.000755 sec
wait id: 54 p1: ''=0x22
p2: ''=0x2ca800
p3: ''=0x2010008
}
and is blocked by 'instance: 2, os id: 45767, session id: 1395',
which is a member of 'Chain 1'.
Chain 7 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 7 Signature Hash: 0x7aa23cc3
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 8:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 53264
process id: 112, oracle@mqdc-dbsr-ordb02
session id: 1155
session serial #: 17936
}
is waiting for 'enq: TX - index contention' with wait info:
{
p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
time in wait: 1281 min 22 sec
timeout after: never
wait id: 44
blocking: 0 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT,EVENT_LON_2000,EVENT_LAT_2000) VALUES ('00120190508000014
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktcwit1()+330<-ktbgtl0()+1250<-kdiins0()+3521<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleRowLoad()+366<-qerltcFetch()+379<-insexe()+1106<-opiexe()+4683<-kpoal8()+2288<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+11
wait history:
* time between current wait and wait #1: 0.001860 sec
1. event: 'rdbms ipc reply'
time waited: 0.000167 sec
wait id: 43 p1: 'from_process'=0x25
p2: 'timeout'=0x7fffffff
* time between wait #1 and #2: 0.000040 sec
2. event: 'rdbms ipc reply'
time waited: 0.000112 sec
wait id: 42 p1: 'from_process'=0x25
p2: 'timeout'=0x7fffffff
* time between wait #2 and #3: 0.000057 sec
3. event: 'rdbms ipc reply'
time waited: 0.000233 sec
wait id: 41 p1: 'from_process'=0x25
p2: 'timeout'=0x7fffffff
}
and is blocked by 'instance: 2, os id: 45767, session id: 1395',
which is a member of 'Chain 1'.
Chain 8 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 8 Signature Hash: 0x7aa23cc3
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 9:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 58058
process id: 116, oracle@mqdc-dbsr-ordb02
session id: 1249
session serial #: 7737
}
is waiting for 'enq: TX - index contention' with wait info:
{
p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
time in wait: 1268 min 56 sec
timeout after: never
wait id: 42
blocking: 0 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT,EVENT_LON_2000,EVENT_LAT_2000) VALUES ('00120190508000014
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktcwit1()+330<-ktbgtl0()+1250<-kdiins0()+3521<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleRowLoad()+366<-qerltcFetch()+379<-insexe()+1106<-opiexe()+4683<-kpoal8()+2288<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+11
wait history:
* time between current wait and wait #1: 0.000382 sec
1. event: 'gc current grant busy'
time waited: 0.000740 sec
wait id: 41 p1: ''=0x22
p2: ''=0x2ca81a
p3: ''=0x2010001
* time between wait #1 and #2: 0.000032 sec
2. event: 'ges message buffer allocation'
time waited: 0.000007 sec
wait id: 40 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #2 and #3: 0.001652 sec
3. event: 'rdbms ipc reply'
time waited: 0.000308 sec
wait id: 39 p1: 'from_process'=0x25
p2: 'timeout'=0x7fffffff
}
and is blocked by 'instance: 2, os id: 45767, session id: 1395',
which is a member of 'Chain 1'.
Chain 9 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 9 Signature Hash: 0x7aa23cc3
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 10:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 72078
process id: 118, oracle@mqdc-dbsr-ordb02
session id: 1299
session serial #: 33264
}
is waiting for 'enq: TX - index contention' with wait info:
{
p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
time in wait: 1166 min 46 sec (last interval)
time in wait: 1230 min 46 sec (total)
timeout after: never
wait id: 273
blocking: 0 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT,EVENT_LON_2000,EVENT_LAT_2000) VALUES ('00120190508000014
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktcwit1()+330<-ktbgtl0()+1250<-kdiins0()+3521<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleRowLoad()+366<-qerltcFetch()+379<-insexe()+1106<-opiexe()+4683<-kpoal8()+2288<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+11
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000010 sec
wait id: 274 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TX - index contention'
time waited: 64 min 0 sec
wait id: 273 p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
* time between wait #2 and #3: 0.000213 sec
3. event: 'gc current grant busy'
time waited: 0.000438 sec
wait id: 272 p1: ''=0x22
p2: ''=0x2ca81c
p3: ''=0x2010001
}
and is blocked by 'instance: 2, os id: 45767, session id: 1395',
which is a member of 'Chain 1'.
Chain 10 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 10 Signature Hash: 0x7aa23cc3
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 11:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 49879
process id: 119, oracle@mqdc-dbsr-ordb02
session id: 1323
session serial #: 30423
}
is waiting for 'enq: TX - index contention' with wait info:
{
p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
time in wait: 1289 min 55 sec
timeout after: never
wait id: 121
blocking: 0 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT,EVENT_LON_2000,EVENT_LAT_2000) VALUES ('00120190508000014
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktcwit1()+330<-ktbgtl0()+1250<-kdiins0()+3521<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleRowLoad()+366<-qerltcFetch()+379<-insexe()+1106<-opiexe()+4683<-kpoal8()+2288<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+11
wait history:
* time between current wait and wait #1: 0.000406 sec
1. event: 'gc current grant busy'
time waited: 0.000840 sec
wait id: 120 p1: ''=0x22
p2: ''=0x2ca81d
p3: ''=0x2010001
* time between wait #1 and #2: 0.000059 sec
2. event: 'ges message buffer allocation'
time waited: 0.000003 sec
wait id: 119 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #2 and #3: 0.000646 sec
3. event: 'SQL*Net message from client'
time waited: 0.002989 sec
wait id: 118 p1: 'driver id'=0x28444553
p2: '#bytes'=0x1
}
and is blocked by 'instance: 2, os id: 45767, session id: 1395',
which is a member of 'Chain 1'.
Chain 11 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 11 Signature Hash: 0x7aa23cc3
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 12:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 57847
process id: 120, oracle@mqdc-dbsr-ordb02
session id: 1347
session serial #: 14816
}
is waiting for 'enq: TX - index contention' with wait info:
{
p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
time in wait: 1269 min 27 sec
timeout after: never
wait id: 44
blocking: 0 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT,EVENT_LON_2000,EVENT_LAT_2000) VALUES ('00120190508000014
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktcwit1()+330<-ktbgtl0()+1250<-kdiins0()+3521<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleRowLoad()+366<-qerltcFetch()+379<-insexe()+1106<-opiexe()+4683<-kpoal8()+2288<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+11
wait history:
* time between current wait and wait #1: 0.000393 sec
1. event: 'gc current grant busy'
time waited: 0.000691 sec
wait id: 43 p1: ''=0x22
p2: ''=0x2ca825
p3: ''=0x2010001
* time between wait #1 and #2: 0.000067 sec
2. event: 'ges message buffer allocation'
time waited: 0.000008 sec
wait id: 42 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #2 and #3: 0.000745 sec
3. event: 'SQL*Net message from client'
time waited: 0.007989 sec
wait id: 41 p1: 'driver id'=0x28444553
p2: '#bytes'=0x1
}
and is blocked by 'instance: 2, os id: 45767, session id: 1395',
which is a member of 'Chain 1'.
Chain 12 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 12 Signature Hash: 0x7aa23cc3
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 13:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 66509
process id: 121, oracle@mqdc-dbsr-ordb02
session id: 1372
session serial #: 16252
}
is waiting for 'enq: TX - index contention' with wait info:
{
p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
time in wait: 1183 min 26 sec (last interval)
time in wait: 1247 min 27 sec (total)
timeout after: never
wait id: 151
blocking: 0 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT,EVENT_LON_2000,EVENT_LAT_2000) VALUES ('00120190508000014
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktcwit1()+330<-ktbgtl0()+1250<-kdiins0()+3521<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleRowLoad()+366<-qerltcFetch()+379<-insexe()+1106<-opiexe()+4683<-kpoal8()+2288<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+11
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000008 sec
wait id: 152 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TX - index contention'
time waited: 64 min 0 sec
wait id: 151 p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
* time between wait #2 and #3: 0.000372 sec
3. event: 'gc current grant busy'
time waited: 0.000632 sec
wait id: 150 p1: ''=0x22
p2: ''=0x2ca82d
p3: ''=0x2010001
}
and is blocked by 'instance: 2, os id: 45767, session id: 1395',
which is a member of 'Chain 1'.
Chain 13 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 13 Signature Hash: 0x7aa23cc3
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 14:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 52656
process id: 63, oracle@mqdc-dbsr-ordb02
session id: 1513
session serial #: 12589
}
is waiting for 'enq: TX - index contention' with wait info:
{
p1: 'name|mode'=0x54580004
p2: 'usn<<16 | slot'=0x13000f
p3: 'sequence'=0xe9f9b2e
time in wait: 1282 min 45 sec
timeout after: never
wait id: 136
blocking: 0 sessions
current sql: INSERT INTO IMD_JC_DAILY_OPER (EVENT_ID,DU_ID,EVENT_NAME,DO_RECEPT_TIME,DO_REPOTER,DO_SURE_TIME,DO_DIRCT,DO_STAKE_STR,DO_STAKE,DO_STAKE_OFFSET,DO_CONT_DESC,DO_REGISTER,DO_STATUS,DO_PARTY,DO_PHONE,DO_EVENT_TYPE,DO_REASON,DO_CAR_DEP,DO_CR_CAR_DESC,DO_CR_STATEMENT,EVENT_LON_2000,EVENT_LAT_2000) VALUES ('00120190508000014
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktcwit1()+330<-ktbgtl0()+1250<-kdiins0()+3521<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-qerltcSingleRowLoad()+366<-qerltcFetch()+379<-insexe()+1106<-opiexe()+4683<-kpoal8()+2288<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+11
wait history:
* time between current wait and wait #1: 0.000427 sec
1. event: 'gc current grant busy'
time waited: 0.000729 sec
wait id: 135 p1: ''=0x22
p2: ''=0x2ca860
p3: ''=0x2010001
* time between wait #1 and #2: 0.000023 sec
2. event: 'ges message buffer allocation'
time waited: 0.000006 sec
wait id: 134 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #2 and #3: 0.000129 sec
3. event: 'gc current grant busy'
time waited: 0.000814 sec
wait id: 133 p1: ''=0x22
p2: ''=0x2ca800
p3: ''=0x2010008
}
and is blocked by 'instance: 2, os id: 45767, session id: 1395',
which is a member of 'Chain 1'.
Chain 14 Signature: 'enq: TT - contention'<='enq: TX - index contention'
Chain 14 Signature Hash: 0x7aa23cc3
-------------------------------------------------------------------------------
===============================================================================
Sessions in an involuntary wait or not in a wait:
-------------------------------------------------------------------------------
Chain 15:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 51133
process id: 129, oracle@mqdc-dbsr-ordb02 (J002)
session id: 26
session serial #: 25617
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 3 min 57 sec (last interval)
time in wait: 19 min 58 sec (total)
timeout after: never
wait id: 125
blocking: 0 sessions
current sql: <none>
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-kttget_alnq()+196<-ktsstrn_segment()+624<-kkbtis_trunc_idx_seg()+564<-kkbindl()+1040<-kkbind()+173<-kkbtrn()+1890<-opiexe()+17971<-opiosq0()+3942<-opipls()+11043<-opiodr()+1149<-__PGOSF141_rpidrus()+211<-skgmstack()+148<-rpiswu2()+617<-rpidrv()+1347<-psddr0()+464<-psdnal()+462<-pevm_EXIM()
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000003 sec
wait id: 126 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 1 sec
wait id: 125 p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000289 sec
3. event: 'db file sequential read'
time waited: 0.008476 sec
wait id: 124 p1: 'file#'=0x6
p2: 'block#'=0xb42
p3: 'blocks'=0x1
}
Chain 15 Signature: 'enq: TT - contention'
Chain 15 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 16:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 75853
process id: 68, oracle@mqdc-dbsr-ordb02
session id: 100
session serial #: 1548
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 4464 min 44 sec
timeout after: never
wait id: 7008
blocking: 0 sessions
current sql: Insert into IMD_JC_LIVE_TRAFFIC (LT_ID,LT_datetime) values (lwltid.nextval, to_date('2019-05-06 08:32:00','yyyy-MM-dd hh24:mi:ss'))
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktsx_starttx()+92<-ktspnr_next()+554<-ktrsexec()+502<-ktspbmphwm()+1323<-ktspmvhwm()+47<-ktsp_bump_hwm()+212<-ktspgsp_main()+1508<-kdisnew()+276<-kdisnewle()+115<-kdisle()+3777<-kdiins0()+40810<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-__PGOSF635_qe
wait history:
* time between current wait and wait #1: 0.000163 sec
1. event: 'db file sequential read'
time waited: 0.012487 sec
wait id: 7007 p1: 'file#'=0xd
p2: 'block#'=0x284881
p3: 'blocks'=0x1
* time between wait #1 and #2: 0.000055 sec
2. event: 'gc cr grant 2-way'
time waited: 0.000327 sec
wait id: 7006 p1: ''=0xd
p2: ''=0x284881
p3: ''=0x8
* time between wait #2 and #3: 0.000037 sec
3. event: 'ges message buffer allocation'
time waited: 0.000003 sec
wait id: 7005 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
}
Chain 16 Signature: 'enq: TT - contention'
Chain 16 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 17:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 33405
process id: 133, oracle@mqdc-dbsr-ordb02 (W004)
session id: 123
session serial #: 10866
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 12806 min 31 sec
timeout after: never
wait id: 49
blocking: 0 sessions
current sql: <none>
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktte_ts_extend()+391<-ktsj_execute_task()+556<-ktsj_slave_main()+1252<-ksvrdp()+1737<-opirip()+735<-opidrv()+603<-sou2o()+103<-opimai_real()+266<-ssthrdmain()+214<-main()+201<-__libc_start_main()+253<-_start()+36
wait history:
* time between current wait and wait #1: 0.000636 sec
1. event: 'Space Manager: slave idle wait'
time waited: 3.940856 sec
wait id: 48 p1: 'Slave ID'=0x4
* time between wait #1 and #2: 0.000073 sec
2. event: 'Space Manager: slave idle wait'
time waited: 5.000933 sec
wait id: 47 p1: 'Slave ID'=0x4
* time between wait #2 and #3: 0.000023 sec
3. event: 'Space Manager: slave idle wait'
time waited: 5.000898 sec
wait id: 46 p1: 'Slave ID'=0x4
}
Chain 17 Signature: 'enq: TT - contention'
Chain 17 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 18:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 75895
process id: 85, oracle@mqdc-dbsr-ordb02
session id: 506
session serial #: 52669
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 4429 min 23 sec (last interval)
time in wait: 4445 min 23 sec (total)
timeout after: never
wait id: 34788
blocking: 0 sessions
current sql: Insert into IMD_JC_FLOW_msm_M (M_ID,ms_id,dir_id,M_DATETIME,CAR_MODEL,M_FLOW,m_avg_spped) values (smsmmid.nextval,17,2, to_date('2019-05-05 23:59:00','yyyy-MM-dd hh24:mi:ss'),8,1358,80.01)
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktsx_starttx()+92<-ktspnr_next()+554<-ktrsexec()+502<-ktspbmphwm()+1323<-ktspmvhwm()+47<-ktsp_bump_hwm()+212<-ktspgsp_main()+1508<-kdisnew()+276<-kdisnewle()+115<-kdisle()+3777<-kdiins0()+40810<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-__PGOSF635_qe
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000003 sec
wait id: 34789 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 34788 p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000131 sec
3. event: 'db file sequential read'
time waited: 0.019141 sec
wait id: 34787 p1: 'file#'=0x17
p2: 'block#'=0x18e280
p3: 'blocks'=0x1
}
Chain 18 Signature: 'enq: TT - contention'
Chain 18 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 19:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 21151
process id: 92, oracle@mqdc-dbsr-ordb02
session id: 674
session serial #: 37895
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540003
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 12836 min 34 sec (last interval)
time in wait: 12852 min 34 sec (total)
timeout after: never
wait id: 2960
blocking: 0 sessions
current sql: ALTER TABLESPACE "TBS_IMDNH" ADD DATAFILE '+DATADG1/imddb/datafile/tbs_imdnh24.dbf' REUSE AUTOEXTEND ON
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-atsdrv()+2046<-opiexe()+16187<-opiosq0()+3942<-kpooprx()+295<-kpoal8()+779<-opiodr()+1149<-ttcpip()+1251<-opitsk()+1633<-opiino()+958<-opiodr()+1149<-opidrv()+570<-sou2o()+103<-opimai_real()+133<-ssthrdmain()+214<-main()+201<-__libc_start_main()+253<-_start()+36
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000004 sec
wait id: 2961 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 2960 p1: 'name|mode'=0x54540003
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000021 sec
3. event: 'ges message buffer allocation'
time waited: 0.000002 sec
wait id: 2959 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
}
Chain 19 Signature: 'enq: TT - contention'
Chain 19 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 20:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 8112
process id: 93, oracle@mqdc-dbsr-ordb02
session id: 699
session serial #: 44228
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 12852 min 26 sec
timeout after: never
wait id: 22169854
blocking: 0 sessions
current sql: insert into IMD_JC_FLOW_HINGE_B_M1_XT(REC_ID,BA_CODE,HIN_CODE,DIR_ID,IO_ID,CAR_MODEL,LANE,UPDATE_TIME,START_TIME,END_TIME,FLOW_MAC,AVG_SPEED)
values
(SEQ_JCFLOW_H_B_XT.NEXTVAL,:1 ,:2 ,:3 ,:4 ,:5 ,:6 ,:7 ,:8 ,:9 ,:10 ,:11 )
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktsx_starttx()+92<-ktspnr_next()+554<-ktrsexec()+502<-ktspbmphwm()+1323<-ktspmvhwm()+47<-ktsp_bump_hwm()+212<-ktspgsp_main()+1508<-kdisnew()+276<-kdisnewle()+115<-kdisle()+3777<-kdiins0()+40810<-kdiinsp()+116<-kauxsin()+1686<-qesltcLoadIndexList()+760<-qesltcLoadIndexes()+55<-__PGOSF635_qe
wait history:
* time between current wait and wait #1: 0.000132 sec
1. event: 'db file sequential read'
time waited: 0.000134 sec
wait id: 22169853 p1: 'file#'=0x4
p2: 'block#'=0x5889
p3: 'blocks'=0x1
* time between wait #1 and #2: 0.000152 sec
2. event: 'db file sequential read'
time waited: 0.000200 sec
wait id: 22169852 p1: 'file#'=0x4
p2: 'block#'=0x5888
p3: 'blocks'=0x1
* time between wait #2 and #3: 0.000438 sec
3. event: 'row cache lock'
time waited: 0.000061 sec
wait id: 22169851 p1: 'cache id'=0xd
p2: 'mode'=0x0
p3: 'request'=0x5
}
Chain 20 Signature: 'enq: TT - contention'
Chain 20 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 21:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 24943
process id: 94, oracle@mqdc-dbsr-ordb02 (W001)
session id: 724
session serial #: 26451
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 12820 min 32 sec (last interval)
time in wait: 12836 min 32 sec (total)
timeout after: never
wait id: 49
blocking: 0 sessions
current sql: <none>
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktte_ts_extend()+391<-ktsj_execute_task()+556<-ktsj_slave_main()+1252<-ksvrdp()+1737<-opirip()+735<-opidrv()+603<-sou2o()+103<-opimai_real()+266<-ssthrdmain()+214<-main()+201<-__libc_start_main()+253<-_start()+36
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000016 sec
wait id: 50 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 49 p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000787 sec
3. event: 'Space Manager: slave idle wait'
time waited: 3.322596 sec
wait id: 48 p1: 'Slave ID'=0x1
}
Chain 21 Signature: 'enq: TT - contention'
Chain 21 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 22:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 75907
process id: 101, oracle@mqdc-dbsr-ordb02
session id: 890
session serial #: 5
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 2783 min 43 sec (last interval)
time in wait: 2799 min 44 sec (total)
timeout after: never
wait id: 41115024
blocking: 0 sessions
current sql: Insert into IMD_JC_DC_PROD_STATUS_LOG (rec_ID,data_code,data_name,table_name,total_num,new_in_num,update_time,in_time) values (sijdpslid.nextval, 'PD0087','交调分钟统计数据','imd_jc_flow_in_jd_detail',14855500,4, to_date('2019-05-07 07:30:00','yyyy-MM-dd hh24:mi:ss'), to_date('2019-05-07 12:20:32','yyyy-MM-dd hh24:mi
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktsx_starttx()+92<-ktspnr_next()+554<-ktrsexec()+502<-ktspbmphwm()+1323<-ktspmvhwm()+47<-ktsp_bump_hwm()+212<-ktspgsp_main()+1508<-kdtgsp()+604<-kdtgsph()+2123<-kdtgrs()+273<-kdtInsRow()+1005<-__PGOSF635_qerltcNoKdtBufferedInsRowCBK()+293<-qerltcSingleRowLoad()+681<-qerltcFetch()+379<-inse
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000003 sec
wait id: 41115025 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 41115024 p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000149 sec
3. event: 'db file sequential read'
time waited: 0.008276 sec
wait id: 41115023 p1: 'file#'=0x20
p2: 'block#'=0x240687
p3: 'blocks'=0x1
}
Chain 22 Signature: 'enq: TT - contention'
Chain 22 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 23:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 27664
process id: 117, oracle@mqdc-dbsr-ordb02 (W002)
session id: 1274
session serial #: 47211
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 12826 min 32 sec
timeout after: never
wait id: 49
blocking: 0 sessions
current sql: <none>
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktte_ts_extend()+391<-ktsj_execute_task()+556<-ktsj_slave_main()+1252<-ksvrdp()+1737<-opirip()+735<-opidrv()+603<-sou2o()+103<-opimai_real()+266<-ssthrdmain()+214<-main()+201<-__libc_start_main()+253<-_start()+36
wait history:
* time between current wait and wait #1: 0.000662 sec
1. event: 'Space Manager: slave idle wait'
time waited: 3.545407 sec
wait id: 48 p1: 'Slave ID'=0x2
* time between wait #1 and #2: 0.000029 sec
2. event: 'Space Manager: slave idle wait'
time waited: 5.000943 sec
wait id: 47 p1: 'Slave ID'=0x2
* time between wait #2 and #3: 0.000030 sec
3. event: 'Space Manager: slave idle wait'
time waited: 5.000959 sec
wait id: 46 p1: 'Slave ID'=0x2
}
Chain 23 Signature: 'enq: TT - contention'
Chain 23 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 24:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 16709
process id: 56, oracle@mqdc-dbsr-ordb02 (W000)
session id: 1345
session serial #: 4111
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 12846 min 33 sec
timeout after: never
wait id: 365
blocking: 0 sessions
current sql: <none>
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktte_ts_extend()+391<-ktsj_execute_task()+556<-ktsj_slave_main()+1252<-ksvrdp()+1737<-opirip()+735<-opidrv()+603<-sou2o()+103<-opimai_real()+266<-ssthrdmain()+214<-main()+201<-__libc_start_main()+253<-_start()+36
wait history:
* time between current wait and wait #1: 0.000128 sec
1. event: 'Space Manager: slave idle wait'
time waited: 3.186509 sec
wait id: 364 p1: 'Slave ID'=0x0
* time between wait #1 and #2: 0.000033 sec
2. event: 'Space Manager: slave idle wait'
time waited: 5.001009 sec
wait id: 363 p1: 'Slave ID'=0x0
* time between wait #2 and #3: 0.000027 sec
3. event: 'Space Manager: slave idle wait'
time waited: 5.000890 sec
wait id: 362 p1: 'Slave ID'=0x0
}
Chain 24 Signature: 'enq: TT - contention'
Chain 24 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 25:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 30424
process id: 123, oracle@mqdc-dbsr-ordb02 (W003)
session id: 1417
session serial #: 51284
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 12800 min 31 sec (last interval)
time in wait: 12816 min 31 sec (total)
timeout after: never
wait id: 49
blocking: 0 sessions
current sql: <none>
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktte_ts_extend()+391<-ktsj_execute_task()+556<-ktsj_slave_main()+1252<-ksvrdp()+1737<-opirip()+735<-opidrv()+603<-sou2o()+103<-opimai_real()+266<-ssthrdmain()+214<-main()+201<-__libc_start_main()+253<-_start()+36
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000011 sec
wait id: 50 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 49 p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000692 sec
3. event: 'Space Manager: slave idle wait'
time waited: 3.778618 sec
wait id: 48 p1: 'Slave ID'=0x3
}
Chain 25 Signature: 'enq: TT - contention'
Chain 25 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Chain 26:
-------------------------------------------------------------------------------
Oracle session identified by:
{
instance: 2 (imddb.imddb2)
os id: 75849
process id: 62, oracle@mqdc-dbsr-ordb02
session id: 1492
session serial #: 11684
}
is waiting for 'enq: TT - contention' with wait info:
{
p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
time in wait: 4124 min 0 sec (last interval)
time in wait: 4140 min 0 sec (total)
timeout after: never
wait id: 1158621
blocking: 0 sessions
current sql: Insert into IMD_JC_FLOW_SA_PARK (Sp_ID,SA_ID,sp_DATETIME) values (ssaspid.nextval,111000000000000006, to_date('2019-05-06 14:00:00','yyyy-MM-dd hh24:mi:ss'))
short stack: ksedsts()+461<-ksdxfstk()+32<-ksdxcb()+1782<-sspuser()+112<-__restore_rt()<-semtimedop()+10<-skgpwwait()+156<-ksliwat()+1821<-kslwaitctx()+162<-kjusuc()+3520<-ksipgetctx()+1413<-ksqcmi()+20639<-ksqgtlctx()+3401<-ksqgelctx()+552<-ktsx_starttx()+92<-ktspnr_next()+554<-ktrsexec()+502<-ktspbmphwm()+1323<-ktspmvhwm()+47<-ktsp_bump_hwm()+212<-ktspgsp_main()+1508<-kdtgsp()+604<-kdtgsph()+2123<-kdtgrs()+273<-kdtInsRow()+1005<-__PGOSF635_qerltcNoKdtBufferedInsRowCBK()+293<-qerltcSingleRowLoad()+681<-qerltcFetch()+379<-inse
wait history:
* time between current wait and wait #1: 0.000000 sec
1. event: 'ges message buffer allocation'
time waited: 0.000004 sec
wait id: 1158622 p1: 'pool'=0x0
p2: 'request'=0x1
p3: 'allocated'=0x0
* time between wait #1 and #2: 0.000000 sec
2. event: 'enq: TT - contention'
time waited: 16 min 0 sec
wait id: 1158621 p1: 'name|mode'=0x54540004
p2: 'tablespace ID'=0x6
p3: 'operation'=0x10
* time between wait #2 and #3: 0.000168 sec
3. event: 'db file sequential read'
time waited: 0.010550 sec
wait id: 1158620 p1: 'file#'=0x1d
p2: 'block#'=0x9fc81
p3: 'blocks'=0x1
}
Chain 26 Signature: 'enq: TT - contention'
Chain 26 Signature Hash: 0xae3bac3c
-------------------------------------------------------------------------------
===============================================================================
Extra information that will be dumped at higher levels:
[level 4] : 1 node dumps -- [LEAF] [LEAF_NW]
[level 5] : 26 node dumps -- [NO_WAIT] [INVOL_WT] [SINGLE_NODE] [NLEAF] [SINGLE_NODE_NW]
State of ALL nodes
([nodenum]/cnode/sid/sess_srno/session/ospid/state/[adjlist]):
[25]/2/26/25617/0x7d1037558/51133/SINGLE_NODE/
[74]/2/75/6778/0x701064fc8/55537/NLEAF/[1394]
[99]/2/100/1548/0x7d10545b8/75853/SINGLE_NODE/
[122]/2/123/10866/0x711081ed0/33405/SINGLE_NODE/
[217]/2/218/52069/0x721108058/72076/NLEAF/[1394]
[266]/2/267/6810/0x7410a6c78/56679/NLEAF/[1394]
[505]/2/506/52669/0x7111218e0/75895/SINGLE_NODE/
[673]/2/674/37895/0x711167360/21151/SINGLE_NODE/
[698]/2/699/44228/0x7511750d0/8112/SINGLE_NODE/
[723]/2/724/26451/0x7411636e8/24943/SINGLE_NODE/
[770]/2/771/49073/0x741177bf8/37926/NLEAF/[1394]
[889]/2/890/5/0x72121ea58/75907/SINGLE_NODE/
[937]/2/938/12418/0x7311b2768/53065/NLEAF/[1394]
[1083]/2/1084/49939/0x71120f8c0/53482/NLEAF/[1394]
[1129]/2/1130/12539/0x70121b3d8/55134/NLEAF/[1394]
[1154]/2/1155/17936/0x7d120a9c8/53264/NLEAF/[1394]
[1248]/2/1249/7737/0x7d12333e8/58058/NLEAF/[1394]
[1273]/2/1274/47211/0x73123dc68/27664/SINGLE_NODE/
[1298]/2/1299/33264/0x711269850/72078/NLEAF/[1394]
[1322]/2/1323/30423/0x7d1250448/49879/NLEAF/[1394]
[1344]/2/1345/4111/0x701275368/16709/SINGLE_NODE/
[1346]/2/1347/14816/0x7212db4c8/57847/NLEAF/[1394]
[1371]/2/1372/16252/0x70127deb8/66509/NLEAF/[1394]
[1394]/2/1395/9988/0x73126f1d8/45767/LEAF/
[1416]/2/1417/51284/0x7d1278e68/30424/SINGLE_NODE/
[1491]/2/1492/11684/0x7512bd040/75849/SINGLE_NODE/
[1512]/2/1513/12589/0x7012bade8/52656/NLEAF/[1394]
*** 2019-05-09 10:48:33.467
===============================================================================
END OF HANG ANALYSIS
===============================================================================
*** 2019-05-09 10:48:33.469
===============================================================================
HANG ANALYSIS DUMPS:
oradebug_node_dump_level: 3
===============================================================================
State of LOCAL nodes
([nodenum]/cnode/sid/sess_srno/session/ospid/state/[adjlist]):
[25]/2/26/25617/0x7d1037558/51133/SINGLE_NODE/
[74]/2/75/6778/0x701064fc8/55537/NLEAF/[1394]
[99]/2/100/1548/0x7d10545b8/75853/SINGLE_NODE/
[122]/2/123/10866/0x711081ed0/33405/SINGLE_NODE/
[217]/2/218/52069/0x721108058/72076/NLEAF/[1394]
[266]/2/267/6810/0x7410a6c78/56679/NLEAF/[1394]
[505]/2/506/52669/0x7111218e0/75895/SINGLE_NODE/
[673]/2/674/37895/0x711167360/21151/SINGLE_NODE/
[698]/2/699/44228/0x7511750d0/8112/SINGLE_NODE/
[723]/2/724/26451/0x7411636e8/24943/SINGLE_NODE/
[770]/2/771/49073/0x741177bf8/37926/NLEAF/[1394]
[889]/2/890/5/0x72121ea58/75907/SINGLE_NODE/
[937]/2/938/12418/0x7311b2768/53065/NLEAF/[1394]
[1083]/2/1084/49939/0x71120f8c0/53482/NLEAF/[1394]
[1129]/2/1130/12539/0x70121b3d8/55134/NLEAF/[1394]
[1154]/2/1155/17936/0x7d120a9c8/53264/NLEAF/[1394]
[1248]/2/1249/7737/0x7d12333e8/58058/NLEAF/[1394]
[1273]/2/1274/47211/0x73123dc68/27664/SINGLE_NODE/
[1298]/2/1299/33264/0x711269850/72078/NLEAF/[1394]
[1322]/2/1323/30423/0x7d1250448/49879/NLEAF/[1394]
[1344]/2/1345/4111/0x701275368/16709/SINGLE_NODE/
[1346]/2/1347/14816/0x7212db4c8/57847/NLEAF/[1394]
[1371]/2/1372/16252/0x70127deb8/66509/NLEAF/[1394]
[1394]/2/1395/9988/0x73126f1d8/45767/LEAF/
[1416]/2/1417/51284/0x7d1278e68/30424/SINGLE_NODE/
[1491]/2/1492/11684/0x7512bd040/75849/SINGLE_NODE/
[1512]/2/1513/12589/0x7012bade8/52656/NLEAF/[1394]
No processes qualify for dumping.
===============================================================================
HANG ANALYSIS DUMPS: END
===============================================================================
*** 2019-05-09 10:48:33.469
Oradebug command 'hanganalyze 3' console output:
Hang Analysis in /u01/app/oracle/diag/rdbms/imddb/imddb2/trace/imddb2_ora_51152.trc


我刚刚看到这个问题。TT 锁一般是跨实例的阻塞导致的。通过 GV$LOCK 可以找到源头。
鉴于此问题时间已久,请关闭问题,也可以重新再开一个问题。关于 TT 队列锁,我以前讲过一个案例,参考:
https://pan.baidu.com/s/1bnPkkPp


