19c打补丁时报错Unable to create patchObject

最近体验了一波19c rac，但在2节点打db补丁的时候遇上了一个qp错误，查阅了很多资料、mos但是都没有类似问题处理方法，最后自己找到了解决方法，到这里分享分享。

故障现象

1节点是gi和db的补丁都是已经打上了补丁的。

[grid@node1 OPatch]$ ./opatch lspatches
29851014;ACFS RELEASE UPDATE 19.4.0.0.0 (29851014)
29850993;OCW RELEASE UPDATE 19.4.0.0.0 (29850993)
29834717;Database Release Update : 19.4.0.0.190716 (29834717)
29401763;TOMCAT RELEASE UPDATE 19.0.0.0.0 (29401763)

OPatch succeeded.
复制

[oracle@node1 OPatch]$ ./opatch lspatches
29850993;OCW RELEASE UPDATE 19.4.0.0.0 (29850993)
29834717;Database Release Update : 19.4.0.0.190716 (29834717)

OPatch succeeded.
复制

2节点gi也是成功打上了的，但是在打db的时候遇上问题了（最开始的冲突检查也是通过的），报错如下：

[root@node2 29834717]# /u01/app/oracle/product/19.3.0/dbhome_1/OPatch/opatchauto apply /soft/29708769 -oh /u01/app/oracle/product/19.3.0/dbhome_1
     Execution of [OPatchAutoBinaryAction] patch action failed, check log for more details. Failures:
Patch Target : node2->/u01/app/oracle/product/19.3.0/dbhome_1 Type[rac]
Details: [
---------------------------Patching Failed---------------------------------
.......
Patch: /soft/29708769/29850993
Log: 
**Reason: Failed during listing in Analysis: java.lang.Exception: oracle.opatch.opatchsdk.OPatchException: Unable to create patchObject
Possible causes are:
   ORACLE_HOME/inventory/oneoffs/29834717 is corrupted. java.lang.RuntimeException: No Patch exists,Please check.**    


Patch: /soft/29708769/29834717
Log: 
Reason: Failed during listing in Analysis: java.lang.Exception: oracle.opatch.opatchsdk.OPatchException: Unable to create patchObject
Possible causes are:
   ORACLE_HOME/inventory/oneoffs/29834717 is corrupted. java.lang.RuntimeException: No Patch exists,Please check. 

After fixing the cause of failure Run opatchauto resume

]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.

OPatchauto session completed at Mon Sep  2 22:39:25 2019
Time taken to complete the session 1 minute, 14 seconds

 opatchauto failed with error code 42
复制

故障处理

在处理之前尝试过很多处理方法（重新打2节点补丁、重新上传opatch、重新解压补丁包和重建中央目录），都没有成功。

最后找到了问题的根源，是2节点$ORACLE_HOME/inventory/oneoffs/下的文件有问题。也就是说是Local Inventory出了问题。

将1节点正确的$ORACLE_HOME/inventory/oneoffs/下的文件复制到2节点：

[oracle@node1 oneoffs]$ scp * node2:$ORACLE_HOME/inventory/oneoffs/
复制

2节点验证补丁集：

[oracle@node2 OPatch]$ ./opatch lspatches
29850993;OCW RELEASE UPDATE 19.4.0.0.0 (29850993)
29834717;Database Release Update : 19.4.0.0.190716 (29834717)

OPatch succeeded.
复制

这样复制会可能会导致缺少某个组件使得oracle出现问题，需要再次打2节点db的补丁。

[root@node2 ~]# /u01/app/oracle/product/19.3.0/dbhome_1/OPatch/opatchauto apply /soft/29708769 -oh /u01/app/oracle/product/19.3.0/dbhome_1
    ......
---------------------------Patching Failed---------------------------------
Command execution failed during patching in home: /u01/app/oracle/product/19.3.0/dbhome_1, host: node2.
Command failed:  /u01/app/oracle/product/19.3.0/dbhome_1/OPatch/opatchauto  apply /soft/29708769 -oh /u01/app/oracle/product/19.3.0/dbhome_1 -target_type rac_database -binary -invPtrLoc /u01/app/oracle/product/19.3.0/dbhome_1/oraInst.loc -jre /u01/app/oracle/product/19.3.0/dbhome_1/OPatch/jre -persistresult /u01/app/oracle/product/19.3.0/dbhome_1/OPatch/auto/dbsessioninfo/sessionresult_node2_rac.ser -analyzedresult /u01/app/oracle/product/19.3.0/dbhome_1/OPatch/auto/dbsessioninfo/sessionresult_analyze_node2_rac.ser
Command failure output: 
==Following patches FAILED in apply:

Patch: /soft/29708769/29850993
Log: /u01/app/oracle/product/19.3.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2019-09-03_01-54-29AM_1.log
Reason: Failed during Patching: oracle.opatch.opatchsdk.OPatchException: ApplySession failed in system modification phase... ‘’'ApplySession::apply failed: java.io.IOException: 
oracle.sysman.oui.patch.PatchException: java.io.FileNotFoundException: /u01/app/oraInventory/ContentsXML/oui-patch.xml (Permission denied)' 

After fixing the cause of failure Run opatchauto resume

]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.

OPatchauto session completed at Tue Sep  3 01:55:33 2019
Time taken to complete the session 4 minutes, 4 seconds

 opatchauto failed with error code 42
复制

这里报错/u01/app/oraInventory/ContentsXML/oui-patch.xml (Permission denied)，对比一节点和二节点文件的权限找到问题。

[root@node1 ~]# ll/u01/app/oraInventory/ContentsXML/oui-patch.xml
-rw-rw---- 1 grid   oinstall 174 Sep  2 20:11 oui-patch.xml

[root@node2 ~]# ll /u01/app/oraInventory/ContentsXML/oui-patch.xml
-rw-r--r-- 1 grid   oinstall 174 Sep  2 20:55 oui-patch.xml
复制

两个节点该文件权限不一致导致。

[root@node2 ContentsXML]# chmod 660 oui-patch.xml
复制

将权限改好之后再次打2节点db补丁。

[root@node2 OPatch]# /u01/app/oracle/product/19.3.0/dbhome_1/OPatch/opatchauto apply /soft/29708769 -oh /u01/app/oracle/product/19.3.0/dbhome_1

OPatchauto session is initiated at Tue Sep  3 19:39:45 2019

......
 
OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:node2
RAC Home:/u01/app/oracle/product/19.3.0/dbhome_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /soft/29708769/29851014
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /soft/29708769/29401763
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /soft/29708769/29850993
Reason: This patch is already been applied, so not going to apply again.

Patch: /soft/29708769/29834717
Reason: This patch is already been applied, so not going to apply again.

OPatchauto session completed at Tue Sep  3 19:41:30 2019
Time taken to complete the session 1 minute, 46 seconds
复制

到这里补丁已经成功打上了！

[oracle@node2 OPatch]$ ./opatch lspatches
29850993;OCW RELEASE UPDATE 19.4.0.0.0 (29850993)
29834717;Database Release Update : 19.4.0.0.190716 (29834717)

OPatch succeeded.
复制