事故场景
场景：级联dg，A>B>C，C库failover后，A库交换机流量被占满，影响A库正常业务。
下面是故障复现及恢复的步骤：
复现步骤
1、不规范配置
级联C库比正常配置多了如下归档传输至A库和B库配置：
ALTER SYSTEM SET log_archive_dest_2='SERVICE=gbkdbdg LGWR ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=gbkdbdg' SCOPE=BOTH SID='*'; ALTER SYSTEM SET log_archive_dest_3='SERVICE=gbkdb LGWR ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=gbkdb' SCOPE=BOTH SID='*';复制
2、B库defer传输至C库
在时间点T1，B库执行defer传输至C库
--B库执行 SQL> alter system set log_archive_dest_state_3=defer; System altered.复制
3、A库正常切换归档
A库模拟业务，切换10次归档
SQL> alter system switch logfile; System altered. SQL> / System altered. SQL> / System altered. SQL> / System altered. SQL> / System altered. SQL> / System altered. SQL> / System altered. SQL> / System altered. SQL> / System altered. SQL> / System altered.复制
4、B库应用正常，但是C库因为没有接收到归档而延迟
--B库 SQL> col name for a24 SQL> col value for a30 SQL> col UNIT for a35 SQL> col TIME_COMPUTED for a40 SQL> col DATUM_TIME for a40 SQL> select * from v$dataguard_stats; NAME VALUE UNIT TIME_COMPUTED DATUM_TIME ------------------------ ------------------------------ ----------------------------------- ---------------------------------------- ---------------------------------------- transport lag +00 00:00:00 day(2) to second(0) interval 03/19/2023 12:42:02 03/19/2023 12:42:01 apply lag +00 00:00:00 day(2) to second(0) interval 03/19/2023 12:42:02 03/19/2023 12:42:01 apply finish time +00 00:00:00.000 day(2) to second(3) interval 03/19/2023 12:42:02 estimated startup time 6 second 03/19/2023 12:42:02 SQL> SQL> select thread#,sequence#,process,status from v$managed_standby; THREAD# SEQUENCE# PROCESS STATUS ---------- ---------- --------------------------- ------------------------------------ 1 34 ARCH CLOSING 1 33 ARCH CLOSING 1 22 ARCH CLOSING 1 32 ARCH CLOSING 0 0 RFS IDLE 0 0 RFS IDLE 1 35 RFS IDLE 1 35 MRP0 APPLYING_LOG 8 rows selected. SQL> set lines 200 SQL> col checkpoint_change# for 999999999999999999999 SQL> SELECT CURRENT_SCN FROM V$DATABASE; CURRENT_SCN ----------- 4569055246 RMAN> list incarnation; using target database control file instead of recovery catalog List of Database Incarnations DB Key Inc Key DB Name DB ID STATUS Reset SCN Reset Time ------- ------- -------- ---------------- --- ---------- ---------- 1 1 GBKDB 845494326 PARENT 1 24-AUG-13 2 2 GBKDB 845494326 PARENT 925702 29-DEC-21 3 3 GBKDB 845494326 ORPHAN 4568599459 13-FEB-23 4 4 GBKDB 845494326 PARENT 4568599516 13-FEB-23 5 5 GBKDB 845494326 ORPHAN 4568606764 13-FEB-23 6 6 GBKDB 845494326 PARENT 4568606764 13-FEB-23 7 7 GBKDB 845494326 PARENT 4568627631 13-FEB-23 8 8 GBKDB 845494326 PARENT 4568628222 13-FEB-23 9 9 GBKDB 845494326 PARENT 4568952372 18-MAR-23 10 10 GBKDB 845494326 CURRENT 4569006964 18-MAR-23 --C库，延迟了10个归档 SQL> set lines 200 SQL> col name for a24 SQL> col value for a30 SQL> col UNIT for a35 SQL> col TIME_COMPUTED for a40 SQL> col DATUM_TIME for a40 SQL> select * from v$dataguard_stats; NAME VALUE UNIT TIME_COMPUTED DATUM_TIME ------------------------ ------------------------------ ----------------------------------- ---------------------------------------- ---------------------------------------- transport lag +00 00:06:44 day(2) to second(0) interval 03/19/2023 12:42:31 03/19/2023 12:35:34 apply lag +00 00:06:44 day(2) to second(0) interval 03/19/2023 12:42:31 03/19/2023 12:35:34 apply finish time day(2) to second(3) interval 03/19/2023 12:42:31 estimated startup time 6 second 03/19/2023 12:42:31 SQL> SQL> select thread#,sequence#,process,status from v$managed_standby; THREAD# SEQUENCE# PROCESS STATUS ---------- ---------- --------------------------- ------------------------------------ 1 20 ARCH CLOSING 1 23 ARCH CLOSING 0 0 ARCH CONNECTED 1 24 ARCH CLOSING 0 0 RFS IDLE 0 0 RFS IDLE 0 0 RFS IDLE 1 25 MRP0 WAIT_FOR_LOG 0 0 RFS IDLE 0 0 RFS IDLE 0 0 RFS IDLE 11 rows selected. SQL> SELECT CURRENT_SCN FROM V$DATABASE; CURRENT_SCN ----------- 4569051565复制
5、C库failover
故障发生：
B库因为收到resetlogs_id，B库的数据库化身发生变化，MRP异常；因此二次请求A库传输归档，从defer到C库之后，主库新增的10个归档。对于生产环境，如果相差的归档较多，归档日志一起重新传输，占用了交换机流量，影响A库的业务。
SQL> alter database recover managed standby database finish; Database altered. SQL> alter database commit to switchover to primary with session shutdown; Database altered. SQL> shutdown abort ORACLE instance shut down. SQL> startup ORACLE instance started. Total System Global Area 626327552 bytes Fixed Size 2255832 bytes Variable Size 234882088 bytes Database Buffers 381681664 bytes Redo Buffers 7507968 bytes Database mounted. Database opened. SQL> --C库日志，因为配置问题，C库resetlog后的日志传输到备库B库 Sun Mar 19 12:28:36 2023 RFS[6]: Assigned to RFS process 3080 RFS[6]: Opened log for thread 1 sequence 21 dbid 845494326 branch 1131820798 Archived Log entry 77 added for thread 1 sequence 21 rlc 1131820798 ID 0x34bcb66a dest 3: Media Recovery Log /u01/app/oracle/gbkdrarch/1_21_1131820798.dbf Media Recovery Log /u01/app/oracle/gbkdrarch/1_22_1131820798.dbf Media Recovery Log /u01/app/oracle/gbkdrarch/1_23_1131820798.dbf Media Recovery Waiting for thread 1 sequence 24 Sun Mar 19 12:28:51 2023 RFS[6]: Selected log 11 for thread 1 sequence 24 dbid 845494326 branch 1131820798 Sun Mar 19 12:28:51 2023 Archived Log entry 78 added for thread 1 sequence 24 ID 0x34bcb66a dest 1: Sun Mar 19 12:28:51 2023 Media Recovery Log /u01/app/oracle/gbkdrarch/1_24_1131820798.dbf Media Recovery Waiting for thread 1 sequence 25 Sun Mar 19 12:47:33 2023 alter database recover managed standby database finish Terminal Recovery: Stopping real time apply Sun Mar 19 12:47:33 2023 MRP0: Background Media Recovery cancelled with status 16037 Errors in file /u01/app/oracle/diag/rdbms/gbkdbdr/gbkdbdr/trace/gbkdbdr_pr00_2898.trc: ORA-16037: user requested cancel of managed recovery operation Managed Standby Recovery not using Real Time Apply Recovery interrupted! Sun Mar 19 12:47:33 2023 MRP0: Background Media Recovery process shutdown (gbkdbdr) Terminal Recovery: Stopped real time apply All dispatchers and shared servers shutdown CLOSE: killing server sessions. CLOSE: all sessions shutdown successfully. Sun Mar 19 12:47:34 2023 SMON: disabling cache recovery Attempt to do a Terminal Recovery (gbkdbdr) Media Recovery Start: Managed Standby Recovery (gbkdbdr) started logmerger process Sun Mar 19 12:47:34 2023 Managed Standby Recovery not using Real Time Apply Parallel Media Recovery started with 2 slaves Media Recovery Waiting for thread 1 sequence 25 Killing 6 processes with pids 2890,2894,2892,3066,3074,3080 (all RFS, wait for I/O) in order to disallow current and future RFS connections. Requested by OS process 3269 Sun Mar 19 12:47:37 2023 idle dispatcher 'D000' terminated, pid = (17, 1) Begin: Standby Redo Logfile archival End: Standby Redo Logfile archival Terminal Recovery timestamp is '03/19/2023 12:47:38' Terminal Recovery: applying standby redo logs. Terminal Recovery: thread 1 seq# 25 redo required Media Recovery Waiting for thread 1 sequence 25 Terminal Recovery: End-Of-Redo log allocation MRP: Validating standby redo logfile 11 Media Recovery Log /u01/app/oracle/oradata/gbkdbdr/sredo11.log Terminal Recovery: log 11 reserved for thread 1 sequence 25 Recovery of Online Redo Log: Thread 1 Group 11 Seq 25 Reading mem 0 Mem# 0: /u01/app/oracle/oradata/gbkdbdr/sredo11.log Identified End-Of-Redo (failover) for thread 1 sequence 25 at SCN 0xffff.ffffffff Incomplete Recovery applied until change 4569051567 time 03/19/2023 12:28:49 Media Recovery Complete (gbkdbdr) Terminal Recovery: successful completion Sun Mar 19 12:51:00 2023 VKRM started with pid=27, OS id=3415 FAL[server, ARC3]: Error 16009 creating remote archivelog file 'gbkdb' FAL[server, ARC3]: FAL archive failed, see trace file. ARCH: FAL archive failed. Archiver continuing ORACLE Instance gbkdbdr - Archival Error. Archiver continuing. Sun Mar 19 12:51:04 2023 Thread 1 advanced to log sequence 3 (LGWR switch) Current log# 3 seq# 3 mem# 0: /u01/app/oracle/oradata/gbkdbdr/redo03.log Archived Log entry 83 added for thread 1 sequence 2 ID 0x34bc8f2c dest 1: ARC2: Standby redo logfile selected for thread 1 sequence 2 for destination LOG_ARCHIVE_DEST_2 ****************************************************************** LGWR: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2 ****************************************************************** LNS: Standby redo logfile selected for thread 1 sequence 3 for destination LOG_ARCHIVE_DEST_2 --B库日志，因为收到resetlogs_id，B库的数据库化身发生变化，MRP异常；并且二次请求A库传输归档，从defer到C库之后，主库新增的10个归档 Sun Mar 19 12:50:57 2023 RFS[4]: Assigned to RFS process 3401 RFS[4]: New Archival REDO Branch: 1131886078 Current: 1131820798 RFS[4]: Selected log 13 for thread 1 sequence 1 dbid 845494326 branch 1131886078 A new recovery destination branch has been registered RFS[4]: Standby in the future of new recovery destinationBranch(resetlogs_id) 1131886078 Incomplete Recovery SCN: 4569052162 Resetlogs SCN: 4569051568 Standby Became Primary SCN: 4569051565 Flashback database to SCN 4569051565 to follow new branch Flashback database to SCN 4569051565 to follow new branch RFS[4]: New Archival REDO Branch(resetlogs_id): 1131886078 Prior: 1131820798 RFS[4]: Archival Activation ID: 0x34bc8f2c Current: 0x34bcb66a RFS[4]: Effect of primary database OPEN RESETLOGS RFS[4]: Managed Standby Recovery process is active RFS[4]: Incarnation entry added for Branch(resetlogs_id): 1131886078 (gbkdb) Sun Mar 19 12:50:57 2023 Setting recovery target incarnation to 11 Sun Mar 19 12:50:57 2023 MRP0: Incarnation has changed! Retry recovery... Errors in file /u01/app/oracle/diag/rdbms/gbkdbdg/gbkdb/trace/gbkdb_pr00_3082.trc: ORA-19906: recovery target incarnation changed during recovery Managed Standby Recovery not using Real Time Apply Sun Mar 19 12:50:57 2023 Archived Log entry 229 added for thread 1 sequence 1 ID 0x34bc8f2c dest 1: Recovery interrupted! Recovered data files to a consistent state at change 4569055775 Sun Mar 19 12:50:58 2023 started logmerger process Sun Mar 19 12:50:58 2023 Managed Standby Recovery starting Real Time Apply Warning: Recovery target destination is in a sibling branch of the controlfile checkpoint. Recovery will only recover changes to datafiles. Datafile 1 (ckpscn 4569055775) is orphaned on incarnation#=10 MRP0: Detected orphaned datafiles! Recovery will possibly be retried after flashback... Errors in file /u01/app/oracle/diag/rdbms/gbkdbdg/gbkdb/trace/gbkdb_pr00_3407.trc: ORA-19909: datafile 1 belongs to an orphan incarnation ORA-01110: data file 1: '/u01/app/oracle/oradata/gbkdbdg/system01.dbf' Managed Standby Recovery not using Real Time Apply Recovery Slave PR00 previously exited with exception 19909 RFS[4]: Opened log for thread 1 sequence 25 dbid 845494326 branch 1131820798 Archived Log entry 230 added for thread 1 sequence 25 rlc 1131820798 ID 0x34bcb66a dest 2: RFS[4]: Selected log 13 for thread 1 sequence 2 dbid 845494326 branch 1131886078 Sun Mar 19 12:51:04 2023 Archived Log entry 231 added for thread 1 sequence 2 ID 0x34bc8f2c dest 1: Sun Mar 19 12:51:04 2023 Primary database is in MAXIMUM PERFORMANCE mode RFS[5]: Assigned to RFS process 3425 RFS[5]: Selected log 13 for thread 1 sequence 3 dbid 845494326 branch 1131886078 Sun Mar 19 12:51:19 2023 MRP0: Background Media Recovery process shutdown (gbkdb) Sun Mar 19 12:51:53 2023 RFS[6]: Assigned to RFS process 3100 RFS[6]: Opened log for thread 1 sequence 27 dbid 845494326 branch 1131820798 Archived Log entry 232 added for thread 1 sequence 27 rlc 1131820798 ID 0x34bcb66a dest 2: RFS[6]: Opened log for thread 1 sequence 28 dbid 845494326 branch 1131820798 Archived Log entry 233 added for thread 1 sequence 28 rlc 1131820798 ID 0x34bcb66a dest 2: RFS[6]: Opened log for thread 1 sequence 29 dbid 845494326 branch 1131820798 Archived Log entry 234 added for thread 1 sequence 29 rlc 1131820798 ID 0x34bcb66a dest 2: RFS[6]: Opened log for thread 1 sequence 30 dbid 845494326 branch 1131820798 Archived Log entry 235 added for thread 1 sequence 30 rlc 1131820798 ID 0x34bcb66a dest 2: Sun Mar 19 12:51:53 2023 RFS[7]: Assigned to RFS process 3439 RFS[7]: Opened log for thread 1 sequence 26 dbid 845494326 branch 1131820798 RFS[6]: Opened log for thread 1 sequence 31 dbid 845494326 branch 1131820798 Archived Log entry 236 added for thread 1 sequence 26 rlc 1131820798 ID 0x34bcb66a dest 2: Archived Log entry 237 added for thread 1 sequence 31 rlc 1131820798 ID 0x34bcb66a dest 2: RFS[6]: Opened log for thread 1 sequence 33 dbid 845494326 branch 1131820798 RFS[7]: Opened log for thread 1 sequence 32 dbid 845494326 branch 1131820798 Archived Log entry 238 added for thread 1 sequence 33 rlc 1131820798 ID 0x34bcb66a dest 2: Archived Log entry 239 added for thread 1 sequence 32 rlc 1131820798 ID 0x34bcb66a dest 2: RFS[6]: Opened log for thread 1 sequence 34 dbid 845494326 branch 1131820798 Archived Log entry 240 added for thread 1 sequence 34 rlc 1131820798 ID 0x34bcb66a dest 2:复制
6、恢复B库
--A库化身情况 RMAN> list incarnation; using target database control file instead of recovery catalog List of Database Incarnations DB Key Inc Key DB Name DB ID STATUS Reset SCN Reset Time ------- ------- -------- ---------------- --- ---------- ---------- 1 1 GBKDB 845494326 PARENT 1 2013-08-24 11:37:30 2 2 GBKDB 845494326 PARENT 925702 2021-12-29 22:29:12 3 3 GBKDB 845494326 ORPHAN 4568599459 2023-02-13 14:53:01 4 4 GBKDB 845494326 PARENT 4568599516 2023-02-13 15:05:13 5 5 GBKDB 845494326 ORPHAN 4568606764 2023-02-13 16:54:05 6 6 GBKDB 845494326 PARENT 4568606764 2023-02-13 16:59:56 7 7 GBKDB 845494326 PARENT 4568627631 2023-02-13 17:03:11 8 8 GBKDB 845494326 PARENT 4568628222 2023-02-13 17:06:18 9 9 GBKDB 845494326 PARENT 4568952372 2023-03-18 12:24:33 10 10 GBKDB 845494326 CURRENT 4569006964 2023-03-18 18:39:58 --B库修改化身为A库的当前化身 RMAN> list incarnation; using target database control file instead of recovery catalog List of Database Incarnations DB Key Inc Key DB Name DB ID STATUS Reset SCN Reset Time ------- ------- -------- ---------------- --- ---------- ---------- 1 1 GBKDB 845494326 PARENT 1 24-AUG-13 2 2 GBKDB 845494326 PARENT 925702 29-DEC-21 3 3 GBKDB 845494326 ORPHAN 4568599459 13-FEB-23 4 4 GBKDB 845494326 PARENT 4568599516 13-FEB-23 5 5 GBKDB 845494326 ORPHAN 4568606764 13-FEB-23 6 6 GBKDB 845494326 PARENT 4568606764 13-FEB-23 7 7 GBKDB 845494326 PARENT 4568627631 13-FEB-23 8 8 GBKDB 845494326 PARENT 4568628222 13-FEB-23 9 9 GBKDB 845494326 PARENT 4568952372 18-MAR-23 10 10 GBKDB 845494326 PARENT 4569006964 18-MAR-23 11 11 GBKDB 845494326 CURRENT 4569051568 19-MAR-23 RMAN> reset database to incarnation 10; RMAN-00571: =========================================================== RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS =============== RMAN-00571: =========================================================== RMAN-03009: failure of reset database command on default channel at 03/19/2023 13:10:01 ORA-19910: can not change recovery target incarnation in control file RMAN> shutdown abort Oracle instance shut down RMAN> startup mount connected to target database (not started) Oracle instance started database mounted Total System Global Area 626327552 bytes Fixed Size 2255832 bytes Variable Size 234882088 bytes Database Buffers 381681664 bytes Redo Buffers 7507968 bytes RMAN> reset database to incarnation 10; database reset to incarnation 10 RMAN> list incarnation; List of Database Incarnations DB Key Inc Key DB Name DB ID STATUS Reset SCN Reset Time ------- ------- -------- ---------------- --- ---------- ---------- 1 1 GBKDB 845494326 PARENT 1 24-AUG-13 2 2 GBKDB 845494326 PARENT 925702 29-DEC-21 3 3 GBKDB 845494326 ORPHAN 4568599459 13-FEB-23 4 4 GBKDB 845494326 PARENT 4568599516 13-FEB-23 5 5 GBKDB 845494326 ORPHAN 4568606764 13-FEB-23 6 6 GBKDB 845494326 PARENT 4568606764 13-FEB-23 7 7 GBKDB 845494326 PARENT 4568627631 13-FEB-23 8 8 GBKDB 845494326 PARENT 4568628222 13-FEB-23 9 9 GBKDB 845494326 PARENT 4568952372 18-MAR-23 10 10 GBKDB 845494326 CURRENT 4569006964 18-MAR-23 11 11 GBKDB 845494326 ORPHAN 4569051568 19-MAR-23 RMAN> SQL> alter database recover managed standby database using current logfile disconnect from session; Database altered. --B库日志，恢复正常，并且即使C库再次传输日志，也不会影响到B库 Sun Mar 19 13:11:01 2023 alter database recover managed standby database using current logfile disconnect from session Attempt to start background Managed Standby Recovery process (gbkdb) Sun Mar 19 13:11:01 2023 MRP0 started with pid=26, OS id=3779 MRP0: Background Managed Standby Recovery process started (gbkdb) started logmerger process Sun Mar 19 13:11:06 2023 Managed Standby Recovery starting Real Time Apply Parallel Media Recovery started with 2 slaves Waiting for all non-current ORLs to be archived... All non-current ORLs have been archived. Media Recovery Log /u01/app/oracle/gbkarch/1_35_1131820798.dbf Media Recovery Waiting for thread 1 sequence 36 (in transit) Recovery of Online Redo Log: Thread 1 Group 12 Seq 36 Reading mem 0 Mem# 0: /u01/app/oracle/oradata/gbkdbdg/sredo12.log Completed: alter database recover managed standby database using current logfile disconnect from session Sun Mar 19 13:12:30 2023 RFS[2]: Selected log 11 for thread 1 sequence 37 dbid 845494326 branch 1131820798 Sun Mar 19 13:12:30 2023 Archived Log entry 242 added for thread 1 sequence 36 ID 0x34bcb66a dest 1: Sun Mar 19 13:12:30 2023 Media Recovery Waiting for thread 1 sequence 37 (in transit) Recovery of Online Redo Log: Thread 1 Group 11 Seq 37 Reading mem 0 Mem# 0: /u01/app/oracle/oradata/gbkdbdg/sredo11.log Sun Mar 19 13:14:22 2023 RFS[3]: Assigned to RFS process 3871 RFS[3]: Opened log for thread 1 sequence 4 dbid 845494326 branch 1131886078 Archived Log entry 243 added for thread 1 sequence 4 rlc 1131886078 ID 0x34bc8f2c dest 2: Sun Mar 19 13:14:22 2023 RFS[4]: Assigned to RFS process 3873 RFS[4]: Selected log 13 for thread 1 sequence 3 dbid 845494326 branch 1131886078 Sun Mar 19 13:14:22 2023 Archived Log entry 244 added for thread 1 sequence 3 ID 0x34bc8f2c dest 1: Sun Mar 19 13:14:26 2023 RFS[5]: Assigned to RFS process 3875 RFS[5]: Selected log 13 for thread 1 sequence 5 dbid 845494326 branch 1131886078 Sun Mar 19 13:14:26 2023 Archived Log entry 245 added for thread 1 sequence 5 ID 0x34bc8f2c dest 1: Sun Mar 19 13:14:26 2023 Primary database is in MAXIMUM PERFORMANCE mode RFS[6]: Assigned to RFS process 3877 RFS[6]: Selected log 13 for thread 1 sequence 6 dbid 845494326 branch 1131886078 Sun Mar 19 13:14:31 2023 RFS[2]: Selected log 12 for thread 1 sequence 38 dbid 845494326 branch 1131820798 Archived Log entry 246 added for thread 1 sequence 37 ID 0x34bcb66a dest 1: Sun Mar 19 13:14:31 2023 Media Recovery Waiting for thread 1 sequence 38 (in transit) Recovery of Online Redo Log: Thread 1 Group 12 Seq 38 Reading mem 0 Mem# 0: /u01/app/oracle/oradata/gbkdbdg/sredo12.log复制
7、为避免再次出现问题，去掉C库异常配置
SQL> ALTER SYSTEM SET log_archive_dest_2=''; System altered. SQL> ALTER SYSTEM SET log_archive_dest_3=''; System altered.复制
8、闪回恢复C库为级联DG备库
--C库执行 SQL> SELECT CURRENT_SCN,STANDBY_BECAME_PRIMARY_SCN FROM V$DATABASE; CURRENT_SCN STANDBY_BECAME_PRIMARY_SCN ----------- -------------------------- 4569055074 4569051565 SQL> select oldest_flashback_scn, to_char(oldest_flashback_time,'yyyy-mm-dd HH24:mi:ss') oldest_flashback_time from v$flashback_database_log; OLDEST_FLASHBACK_SCN OLDEST_FLASHBACK_TIME -------------------- --------------------------------------------------------- 4568952074 2023-03-18 12:18:42 SQL> startup mount force ORACLE instance started. Total System Global Area 626327552 bytes Fixed Size 2255832 bytes Variable Size 234882088 bytes Database Buffers 381681664 bytes Redo Buffers 7507968 bytes Database mounted. SQL> flashback database to scn 4569051565; Flashback complete. SQL> alter database open; alter database open * ERROR at line 1: ORA-01589: must use RESETLOGS or NORESETLOGS option for database open SQL> alter database convert to physical standby; Database altered. SQL> shutdown abort ORACLE instance shut down. SQL> startup ORACLE instance started. Total System Global Area 626327552 bytes Fixed Size 2255832 bytes Variable Size 234882088 bytes Database Buffers 381681664 bytes Redo Buffers 7507968 bytes Database mounted. Database opened. SQL> SQL> alter database recover managed standby database using current logfile disconnect from session; Database altered. --B库enable归档传输到C库 SQL> alter system set log_archive_dest_state_3=enable; System altered. --C库应用日志报错 Sun Mar 19 13:33:22 2023 Managed Standby Recovery starting Real Time Apply Parallel Media Recovery started with 2 slaves Waiting for all non-current ORLs to be archived... All non-current ORLs have been archived. Media Recovery Log /u01/app/oracle/gbkdrarch/1_24_1131820798.dbf Media Recovery Log /u01/app/oracle/gbkdrarch/1_25_1131820798.dbf Identified End-Of-Redo (failover) for thread 1 sequence 25 at SCN 0x1.105631af Resetting standby activation ID 884782698 (0x34bcb66a) Incomplete Recovery applied until change 4569051567 time 03/19/2023 12:47:38 MRP0: Background Media Recovery applied all available redo. Recovery will be restarted once new redo branch is registered Errors in file /u01/app/oracle/diag/rdbms/gbkdbdr/gbkdbdr/trace/gbkdbdr_pr00_4507.trc: ORA-19906: recovery target incarnation changed during recovery Managed Standby Recovery not using Real Time Apply Recovery interrupted! RMAN> list incarnation ; using target database control file instead of recovery catalog List of Database Incarnations DB Key Inc Key DB Name DB ID STATUS Reset SCN Reset Time ------- ------- -------- ---------------- --- ---------- ---------- 1 1 GBKDB 845494326 PARENT 1 24-AUG-13 2 2 GBKDB 845494326 PARENT 925702 29-DEC-21 3 3 GBKDB 845494326 ORPHAN 4568599459 13-FEB-23 4 4 GBKDB 845494326 PARENT 4568599516 13-FEB-23 5 5 GBKDB 845494326 ORPHAN 4568606764 13-FEB-23 6 6 GBKDB 845494326 PARENT 4568606764 13-FEB-23 7 7 GBKDB 845494326 PARENT 4568627631 13-FEB-23 8 8 GBKDB 845494326 PARENT 4568628222 13-FEB-23 9 9 GBKDB 845494326 PARENT 4568952372 18-MAR-23 10 10 GBKDB 845494326 CURRENT 4569006964 18-MAR-23 11 11 GBKDB 845494326 ORPHAN 4569051568 19-MAR-23 RMAN> --此时的问题在于25号归档被C库重新构建后传输到了B库，并非A库的25号归档。尝试从A库拷贝25号归档到C库注册后，C库MRP应用日志正常。 25号归档报错选择了重新搭建，下面测试为卡住66号归档后恢复的日志显示： --B库，failover之前的66号归档 [oracle@linux8 gbkarch]$ ls -l 1_66_1131820798.dbf -rw-r----- 1 oracle oinstall 2077184 Mar 19 16:11 1_66_1131820798.dbf [oracle@linux8 gbkarch]$ ls -l 1_67_1131820798.dbf -rw-r----- 1 oracle oinstall 1536 Mar 19 16:11 1_67_1131820798.dbf [oracle@linux8 gbkarch]$ md5sum 1_66_1131820798.dbf 6b3dd2a27e7f227a1626c4758a35d1b7 1_66_1131820798.dbf [oracle@linux8 gbkarch]$ md5sum 1_67_1131820798.dbf 6a167be35ce82f18e3784d302d9d0174 1_67_1131820798.dbf --C库failover后，B库的66号归档是C库构建出来的Identified End-Of-Redo (failover) 归档 --C库传输自己构建的66号归档给B库 Sun Mar 19 16:15:14 2023 CJQ0 started with pid=29, OS id=11136 PING[ARC2]: Heartbeat failed to connect to standby 'gbkdb'. Error is 16009. ARC1: Error 16009 Creating archive log file to 'gbkdb' ARC1: Standby redo logfile selected for thread 1 sequence 1 for destination LOG_ARCHIVE_DEST_2 Archived Log entry 42 added for thread 1 sequence 1 ID 0x34bced0d dest 1: Sun Mar 19 16:15:16 2023 NSA3 started with pid=27, OS id=11148 Setting Resource Manager plan SCHEDULER[0x32DF]:DEFAULT_MAINTENANCE_PLAN via scheduler window Setting Resource Manager plan DEFAULT_MAINTENANCE_PLAN via parameter Starting background process VKRM Sun Mar 19 16:15:17 2023 VKRM started with pid=28, OS id=11150 Sun Mar 19 16:15:21 2023 Thread 1 advanced to log sequence 3 (LGWR switch) Current log# 3 seq# 3 mem# 0: /u01/app/oracle/oradata/gbkdbdr/redo03.log ARC0: STARTING ARCH PROCESSES Sun Mar 19 16:15:21 2023 ARC4 started with pid=35, OS id=11162 ARC4: Archival started ARC0: STARTING ARCH PROCESSES COMPLETE krsk_srl_archive_int: Enabling archival of deferred physical standby SRLs Archived Log entry 44 added for thread 1 sequence 2 ID 0x34bced0d dest 1: Archived Log entry 45 added for thread 1 sequence 66 ID 0x34bcb66a dest 1: --66号归档已经发生变化 [oracle@linux8 gbkarch]$ ls -l 1_66_1131820798.dbf -rw-r----- 1 oracle oinstall 1024 Mar 19 16:15 1_66_1131820798.dbf [oracle@linux8 gbkarch]$ ls -l 1_67_1131820798.dbf -rw-r----- 1 oracle oinstall 1536 Mar 19 16:15 1_67_1131820798.dbf [oracle@linux8 gbkarch]$ --C库日志应用报错 Sun Mar 19 16:25:55 2023 Managed Standby Recovery starting Real Time Apply Parallel Media Recovery started with 2 slaves Waiting for all non-current ORLs to be archived... All non-current ORLs have been archived. Media Recovery Log /u01/app/oracle/gbkdrarch/1_66_1131820798.dbf Identified End-Of-Redo (failover) for thread 1 sequence 66 at SCN 0x1.10567df0 Resetting standby activation ID 0 (0x0) Incomplete Recovery applied until change 4569071088 time 03/19/2023 16:12:28 MRP0: Background Media Recovery applied all available redo. Recovery will be restarted once new redo branch is registered Errors in file /u01/app/oracle/diag/rdbms/gbkdbdr/gbkdbdr/trace/gbkdbdr_pr00_11620.trc: ORA-19906: recovery target incarnation changed during recovery Managed Standby Recovery not using Real Time Apply Recovery interrupted! --C库注册新拷贝的A库的归档 RMAN> catalog start with '/u01/app/oracle/gbkdrarch/1_66_1131820798.dbf'; --拷贝注册后，C库应用日志正常 Sun Mar 19 16:30:45 2023 Managed Standby Recovery starting Real Time Apply Parallel Media Recovery started with 2 slaves Waiting for all non-current ORLs to be archived... All non-current ORLs have been archived. Media Recovery Log /u01/app/oracle/gbkdrarch/1_66_1131820798.dbf Media Recovery Log /u01/app/oracle/gbkdrarch/1_67_1131820798.dbf Media Recovery Log /u01/app/oracle/gbkdrarch/1_68_1131820798.dbf Media Recovery Log /u01/app/oracle/gbkdrarch/1_69_1131820798.dbf Media Recovery Log /u01/app/oracle/gbkdrarch/1_70_1131820798.dbf Media Recovery Log /u01/app/oracle/gbkdrarch/1_71_1131820798.dbf Media Recovery Log /u01/app/oracle/gbkdrarch/1_72_1131820798.dbf Media Recovery Log /u01/app/oracle/gbkdrarch/1_73_1131820798.dbf Media Recovery Log /u01/app/oracle/gbkdrarch/1_74_1131820798.dbf Media Recovery Log /u01/app/oracle/gbkdrarch/1_75_1131820798.dbf Media Recovery Log /u01/app/oracle/gbkdrarch/1_76_1131820798.dbf Media Recovery Waiting for thread 1 sequence 77复制
最后修改时间：2023-03-23 10:26:51
【版权声明】本文为墨天轮用户原创内容，转载时必须标注文章的来源（墨天轮），文章链接，文章作者等基本信息，否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容，欢迎发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。
文章被以下合辑收录
评论
相关阅读
2025年3月中国数据库排行榜：PolarDB夺魁傲群雄，GoldenDB晋位入三强
墨天轮编辑部
1794次阅读
2025-03-11 17:13:58
【专家有话说第五期】在不同年龄段，DBA应该怎样规划自己的职业发展？
墨天轮编辑部
1270次阅读
2025-03-13 11:40:53
【专家观点】罗敏：从理论到真实SQL，感受DeepSeek如何做性能优化
墨天轮编辑部
1270次阅读
2025-03-06 16:45:38
01. HarmonyOS Next应用开发实践与技术解析
若城
1188次阅读
2025-03-04 21:06:20
DeepSeek R1助力，腾讯AI代码助手解锁音乐创作新
若城
1175次阅读
2025-03-05 09:05:00
03 HarmonyOS Next仪表盘案例详解（二）：进阶篇
若城
1168次阅读
2025-03-04 21:08:36
05 HarmonyOS NEXT高效编程秘籍：Arkts函数调用与声明优化深度解析
若城
1156次阅读
2025-03-04 22:46:06
04 高效HarmonyOS NEXT编程：ArkTS数据结构优化与属性访问最佳实践
若城
1150次阅读
2025-03-04 21:09:35
02 HarmonyOS Next仪表盘案例详解（一）：基础篇
若城
1146次阅读
2025-03-04 21:07:43
06 HarmonyOS Next性能优化之LazyForEach 列表渲染基础与实现详解 （一）
若城
1140次阅读
2025-03-05 21:09:40
