ODAX7-2HA 安装部署文档
本文通过ODA X7-2 HA实施过程进行概要描述,旨在进行分享交流,难免有遗漏,欢迎勘正。同时欢迎加入XAOUG(西安ORACLE用户组),加入方法详见文末彩蛋。
目录
1. 安装概述
1.1 ODAX7-2 HA 架构图
图1 架构图
硬件概况:
ODA X7-2由2台计算节点(Computer Node)和1台存储节点(Disk Array)组成;
Coputure Node对外使用4根10GBase-T(铜缆
)连接外部交换机,每台2根进行双网卡绑定;Coputure Node之间互联使用2根25GbE SFP28(光纤)网络互联;
Coputure Node和Disk Array使用4根线进行互联。
图2 磁盘划分
磁盘概况:
Coputure Node主机层面使用自带磁盘进行RAID冗余,磁盘进行裸机OS安装或KVM(虚拟化)部署,同时也安装ORACLE 软件和ORACLE trace日志等(非redo、archive)。
Disk Array使用ASM进行冗余:
+DATA存放数据文件存储;
+RECO存放归档日志和备份;
+REDO存放在线日志;
+FALSH配备了相应HDDs硬件后,存放热表数据、数据文件、buffer cache等。
1.2 系统参数概要
1.2 .1
IP概要节点名称 | Public IP | Private IP | Virtual IP | Scan IP | ILOM IP |
oak1 | 172.16.10.11 | 192.168.10.11 192.168.11.11 | 172.16.10.13 | 172.16.10.15 | 172.16.75.11 |
oak2 | 172.16.10.12 | 172.16.10.12 172.16.11.12 | 172.16.10.14 | 172.16.75.12 |
网段划分为管理段、业务段、心跳段。
ilom管理段为172.16.10.11、172.16.10.12,
Netmask:255.255.255.0,Gateway:172.16.10.254;
业务段为172.16.10.11、172.16.10.12,
Netmask:255.255.255.0,Gateway:172.16.10.254;
心跳段
为192.168.10.11、172.16.10.12和192.168.11.11、172.16.11.12。1.2.2 orcl数据库概要
Oracle SID:orcl(可部署多个实例)
Oracle version :12.2
Oracle 字符集:ZHS16GBK
Oracle ASM 冗余度:high(数据3份冗余,可用容量约存储总量的1/3)。
1.3 密码概况
类别序号 | 类别 | 分项序号 | 用户名 | 密码 |
1 | ilom | 1 | root | changeme |
2 | 操作系统 | 2 | root | changeme |
3 | grid | ****** | ||
4 | oracle | ****** | ||
3 | 数据库 | 5 | asmadmin | ****** |
6 | sys/system | ****** | ||
4 | mgmt | 7 | oda-admin | Welcome1 |
2. 实施概要
2.1 物理连接线缆
图3 内部连线
Coputure Node对外分别连接每台的NET1、NET2和外部交换机相连,共4根,每台2根进行双网卡绑定;
Coputure Node之间分别使用绿色、黄色线互联相同颜色端口;
Coputure Node和Disk Array使用浅红色、浅蓝色、深红色、深蓝色4根线进行互联。
2.2 ilom IP配置[oak1/oak2]
1) Ilom ip配置
配置oak1管理口地址为172.16.75.11,同样配置另外一台为172.16.75.12。
ORACLESP-1845XD302N login: root
Password:
Detecting screen size; please wait...done
Oracle(R) Integrated Lights Out Manager
Version 4.0.2.20.b r123704
Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved.
Warning: password is set to factory default.
Warning: HTTPS certificate is set to factory default.
Hostname: ORACLESP-1845XD302N
-> cd SP/network/
/SP/network
-> set pendingipaddress=172.16.75.11
Set 'pendingipaddress' to '172.16.75.11'
-> set pendingipgateway=172.16.75.254
Set 'pendingipgateway' to '172.16.75.254'
-> set pendingipnetmask=255.255.255.0
Set 'pendingipnetmask' to '255.255.255.0'
-> set commitpending=true
Set 'commitpending' to 'true'
-> set state=enable
Set 'state' to 'enable' [enabled]
2) 通过SSH方式登陆到ilom ip
ssh 172.16.75.11
username/password:root/changeme
3) Ilom 层切换到系统层
2.3 系统安装[oak1/oak2]
1) 登录ilom web界面
通过web登录(https://172.16.75.11)。用户名和密码:root/changeme
2) 打开远程控制台
3)加载本地iso
4)选择下一次重启从光驱启动,保存设置
5)保存从光驱启动
6)重启系统,保存设置
7)保存重启设置
8)机器重启后,会从镜像光驱引导,不用做操作
9)系统安装进度1
10)系统安装进度2
11)系统安装进度3
左下角会有一个光标在闪烁,大概30分钟左右完成该节点系统安装。
2.4 系统初始化IP [oak1/oak2]
以oak1为例,oak2参考oak1进行配置。
[root@oak1 ~]# opt/oracle/dcs/bin/odacli configure-firstnet
Using bonding public interface (yes/no) [yes]:
Select the Interface to configure the network on (btbond1) [btbond1]:
Configure DHCP on btbond1 (yes/no) [no]:
INFO: You have chosen Static configuration
Use VLAN on btbond1 (yes/no) [no]:
Enter the IP address to configure : 172.16.10.11
Enter the Netmask address to configure : 255.255.255.0
Enter the Gateway address to configure[172.16.10.1] : 172.16.10.254
INFO: Plumbing the IPs now
INFO: Restarting the network
Shutting down interface btbond1: [ OK ]
Shutting down interface em1: [ OK ]
Shutting down interface p1p1: [ OK ]
Shutting down interface p1p2: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface btbond1: Determining if ip address 172.16.10.11 is already in use for device btbond1...
[ OK ]
Bringing up interface em1: [ OK ]
Bringing up interface p1p1: Determining if ip address 192.168.10.11 is already in use for device p1p1...
[ OK ]
Bringing up interface p1p2: Determining if ip address 192.168.11.11 is already in use for device p1p2...
[ OK ]
[root@oak1 ~]#
2.5 上传相关介质
p27119402_122140_Linux-x86-64.zip
p27119393_122140_Linux-x86-64.zip
2.6 GI安装
2.6.1 查看安装版本odacli describe-component
[root@oak1 ~]# opt/oracle/dcs/bin/odacli describe-component
System Version
---------------
12.2.1.4.0
System node Name
---------------
oak1
Local System Version
---------------
12.2.1.4.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK 12.2.1.4.0 up-to-date
DCSAGENT 18.2.1.0.0 up-to-date
ILOM 4.0.2.20.b.r123704 up-to-date
BIOS 41021300 up-to-date
OS 6.9 up-to-date
FIRMWARECONTROLLER 13.00.00.00 up-to-date
FIRMWAREEXPANDER 0309 0306
FIRMWAREDISK A170 a122
System node Name
---------------
oak2
Local System Version
---------------
12.2.1.4.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK 12.2.1.4.0 up-to-date
DCSAGENT 18.2.1.0.0 up-to-date
ILOM 4.0.2.20.b.r123704 up-to-date
BIOS 41021300 up-to-date
OS 6.9 up-to-date
FIRMWARECONTROLLER 13.00.00.00 up-to-date
FIRMWAREEXPANDER 0309 0306
FIRMWAREDISK A170 a122
2.6.2 解压GI(p27119393_122140)
[root@oak1 soft]# unzip p27119393_122140_Linux-x86-64.zip
Archive: p27119393_122140_Linux-x86-64.zip
inflating: README.txt
extracting: odacli-dcs-12.2.1.4.0-180617-GI-12.2.0.1.zip
2.6.3 部署GI(odacli update-repository -f)
[root@oak1 soft]# opt/oracle/dcs/bin/odacli update-repository -f /u01/soft/odacli-dcs-
12.2.1.4.0-180617-GI-12.2.0.1.zip
{
"jobId" : "4b1c68f9-527d-4d70-abec-5d45b672bd95",
"status" : "Created",
"message" : "/u01/soft/odacli-dcs-12.2.1.4.0-180617-GI-12.2.0.1.zip",
"reports" : [ ],
"createTimestamp" : "December 18, 2018 19:27:13 PM PST",
"resourceList" : [ ],
"description" : "Repository Update",
"updatedTime" : "December 18, 2018 19:27:13 PM PST"
}
2.6.4 查看部署进度(odacli describe-job -i)
[root@oak1 ]# opt/oracle/dcs/bin/odacli describe-job -i 4b1c68f9-527d-4d70-abec-5d45b672bd95
Job details
----------------------------------------------------------------
ID: 4b1c68f9-527d-4d70-abec-5d45b672bd95
Description: Repository Update
Status: Success
Created: December 18, 2018 7:27:13 PM PST
Message: u01/soft/odacli-dcs-12.2.1.4.0-180617-GI-12.2.0.1.zip
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Check AvailableSpace December 18, 2018 7:27:13 PM PST December 18, 2018 7:27:13 PM PST Success
Setting up ssh equivalance December 18, 2018 7:27:13 PM PST December 18, 2018 7:27:13 PM PST Success
Copy BundleFile December 18, 2018 7:27:13 PM PST December 18, 2018 7:27:32 PM PST Success
Validating CopiedFile December 18, 2018 7:27:33 PM PST December 18, 2018 7:27:42 PM PST Success
Unzip bundle December 18, 2018 7:27:42 PM PST December 18, 2018 7:28:19 PM PST Success
Unzip bundle December 18, 2018 7:28:21 PM PST December 18, 2018 7:28:58 PM PST Success
Delete PatchBundles December 18, 2018 7:28:58 PM PST December 18, 2018 7:28:59 PM PST Success
Removing ssh keys
2.7 DB安装
2.7.1 解压DB(p27119402_122140)
[root@oak1 soft]# unzip p27119402_122140_Linux-x86-64.zip
Archive: p27119402_122140_Linux-x86-64.zip
extracting: odacli-dcs-12.2.1.4.0-180617-DB-12.2.0.1.zip
replace README.txt? [y]es, [n]o, [A]ll, [N]one, [r]ename: yes
inflating: README.txt
2.7.2 部署DB(odacli update-repository -f)
[root@oak1 ]# opt/oracle/dcs/bin/odacli update-repository -f u01/soft/odacli-dcs-12.2.1.4.0-180617-DB-12.2.0.1.zip
{
"jobId" : "cd3bb654-d3ef-465f-8277-ef77e24f4e6e",
"status" : "Created",
"message" : "/u01/soft/odacli-dcs-12.2.1.4.0-180617-DB-12.2.0.1.zip",
"reports" : [ ],
"createTimestamp" : "December 18, 2018 19:33:02 PM PST",
"resourceList" : [ ],
"description" : "Repository Update",
"updatedTime" : "December 18, 2018 19:33:02 PM PST"
}
2.7.3 查看部署进度(odacli describe-job -i)
[root@oak1 ]# opt/oracle/dcs/bin/odacli describe-job -i cd3bb654-d3ef-465f-8277-ef77e24f4e6e
Job details
----------------------------------------------------------------
ID: cd3bb654-d3ef-465f-8277-ef77e24f4e6e
Description: Repository Update
Status: Success
Created: December 18, 2018 7:33:02 PM PST
Message: u01/soft/odacli-dcs-12.2.1.4.0-180617-DB-12.2.0.1.zip
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Check AvailableSpace December 18, 2018 7:33:02 PM PST December 18, 2018 7:33:02 PM PST Success
Setting up ssh equivalance December 18, 2018 7:33:02 PM PST December 18, 2018 7:33:02 PM PST Success
Copy BundleFile December 18, 2018 7:33:02 PM PST December 18, 2018 7:33:16 PM PST Success
Validating CopiedFile December 18, 2018 7:33:17 PM PST December 18, 2018 7:33:25 PM PST Success
Unzip bundle December 18, 2018 7:33:25 PM PST December 18, 2018 7:33:51 PM PST Success
Unzip bundle December 18, 2018 7:33:53 PM PST December 18, 2018 7:34:21 PM PST Success
Delete PatchBundles December 18, 2018 7:34:21 PM PST December 18, 2018 7:34:21 PM PST Success
Removing ssh keys December 18, 2018 7:34:21 PM PST December 18, 2018 7:34:22 PM PST Success
2.8 登录mgmt创建实例
2.8.1 登陆mgmt创建实例
1) 登录mgmt
https://172.16.10.11:7093/mgmt/index.html,输入用户名和密码oda-admin/******登录。
2)创建应用
3) 输入系统信息
4) 输入主机名等信息
按规划地址填写,以下内容仅供参考。
5) 输入scan信息
按规划地址填写,以下内容仅供参考。
6) 输入实例名字符集等信息
输入实例名、版本、是否使用CDB等数据库信息。
注:主机资源通过需要通过shape进行调整,详见章节3“ODA X7-2 HA OLTP 资源类型表”。
7) 不启用ASR,点击提交
8) Job已经创建
9) 创建过程
2.8.2 查看job(odacli list-jobs)
[root@oak1 soft]# opt/oracle/dcs/bin/odacli list-jobs
ID Description Created Status
---------------------------------------- --------------------------------------------------------------------------- ----------------------------------- ----------
4b1c68f9-527d-4d70-abec-5d45b672bd95 Repository Update December 19, 2018 11:27:13 AM CST Success
cd3bb654-d3ef-465f-8277-ef77e24f4e6e Repository Update December 19, 2018 11:33:02 AM CST Success
b8e83506-98c7-4910-a810-e3a2626b9647 Provisioning service creation December 19, 2018 11:48:41 AM CST Running
当前正在运行。
2.8.3 查看部署进度(odacli describe-job -i)
[root@oak1 soft]# opt/oracle/dcs/bin/odacli describe-job -i b8e83506-98c7-4910-a810-e3a2626b9647
Job details
----------------------------------------------------------------
ID: b8e83506-98c7-4910-a810-e3a2626b9647
Description: Provisioning service creation
Status: Running
Created: December 19, 2018 11:48:41 AM CST
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
OS usergroup 'asmdba'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
OS usergroup 'asmoper'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
OS usergroup 'asmadmin'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
OS usergroup 'dba'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
OS usergroup 'dbaoper'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
OS usergroup 'oinstall'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
OS user 'grid'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
OS user 'oracle'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
SSH equivalance setup December 19, 2018 11:48:44 AM CST December 19, 2018 11:48:44 AM CST Success
Grid home creation December 19, 2018 11:48:54 AM CST December 19, 2018 11:48:54 AM CST Running
Creating GI home directories December 19, 2018 11:48:54 AM CST December 19, 2018 11:48:54 AM CST Success
Cloning Gi home December 19, 2018 11:48:54 AM CST December 19, 2018 11:48:54 AM CST Running
当前已完成和正在运行的进度。
2.8.4 查看部署进度-完成(odacli describe-job -i)
[root@oak1 ~]# /opt/oracle/dcs/bin/odacli describe-job -i b8e83506-98c7-4910-a810-e3a2626b9647
Job details
----------------------------------------------------------------
ID: b8e83506-98c7-4910-a810-e3a2626b9647
Description: Provisioning service creation
Status: Success
Created: December 19, 2018 11:48:41 AM CST
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
OS usergroup 'asmdba'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
OS usergroup 'asmoper'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
OS usergroup 'asmadmin'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
OS usergroup 'dba'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
OS usergroup 'dbaoper'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
OS usergroup 'oinstall'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
OS user 'grid'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
OS user 'oracle'creation December 19, 2018 11:48:43 AM CST December 19, 2018 11:48:43 AM CST Success
SSH equivalance setup December 19, 2018 11:48:44 AM CST December 19, 2018 11:48:44 AM CST Success
Grid home creation December 19, 2018 11:48:54 AM CST December 19, 2018 11:54:35 AM CST Success
Creating GI home directories December 19, 2018 11:48:54 AM CST December 19, 2018 11:48:54 AM CST Success
Cloning Gi home December 19, 2018 11:48:54 AM CST December 19, 2018 11:51:42 AM CST Success
Cloning Gi home December 19, 2018 11:51:43 AM CST December 19, 2018 11:54:32 AM CST Success
Updating GiHome version December 19, 2018 11:54:33 AM CST December 19, 2018 11:54:34 AM CST Success
Updating GiHome version December 19, 2018 11:54:33 AM CST December 19, 2018 11:54:34 AM CST Success
Storage discovery December 19, 2018 11:54:35 AM CST December 19, 2018 12:02:59 PM CST Success
Grid stack creation December 19, 2018 12:02:59 PM CST December 19, 2018 12:35:07 PM CST Success
Configuring GI December 19, 2018 12:02:59 PM CST December 19, 2018 12:03:17 PM CST Success
Running GI root scripts December 19, 2018 12:03:17 PM CST December 19, 2018 12:14:16 PM CST Success
Running GI config assistants December 19, 2018 12:20:19 PM CST December 19, 2018 12:31:23 PM CST Success
Post cluster OAKD configuration December 19, 2018 12:35:07 PM CST December 19, 2018 12:39:33 PM CST Success
Disk group 'RECO'creation December 19, 2018 12:39:41 PM CST December 19, 2018 12:39:54 PM CST Success
Disk group 'REDO'creation December 19, 2018 12:39:54 PM CST December 19, 2018 12:40:05 PM CST Success
Volume 'commonstore'creation December 19, 2018 12:40:05 PM CST December 19, 2018 12:40:36 PM CST Success
ACFS File system 'DATA'creation December 19, 2018 12:40:36 PM CST December 19, 2018 12:40:46 PM CST Success
Database home creation December 19, 2018 12:40:46 PM CST December 19, 2018 12:44:06 PM CST Success
Validating dbHome available space December 19, 2018 12:40:46 PM CST December 19, 2018 12:40:46 PM CST Success
Validating dbHome available space December 19, 2018 12:40:46 PM CST December 19, 2018 12:40:46 PM CST Success
Creating DbHome Directory December 19, 2018 12:40:46 PM CST December 19, 2018 12:40:46 PM CST Success
Extract DB clones December 19, 2018 12:40:46 PM CST December 19, 2018 12:42:46 PM CST Success
Clone Db home December 19, 2018 12:42:46 PM CST December 19, 2018 12:43:49 PM CST Success
Enable DB options December 19, 2018 12:43:49 PM CST December 19, 2018 12:43:59 PM CST Success
Run Root DB scripts December 19, 2018 12:43:59 PM CST December 19, 2018 12:43:59 PM CST Success
Provisioning service creation December 19, 2018 12:44:07 PM CST December 19, 2018 12:52:43 PM CST Success
Database Creation December 19, 2018 12:44:07 PM CST December 19, 2018 12:50:42 PM CST Success
Change permission for xdb wallet files December 19, 2018 12:50:42 PM CST December 19, 2018 12:50:42 PM CST Success
Running DataPatch December 19, 2018 12:52:11 PM CST December 19, 2018 12:52:39 PM CST Success
Running dbms_stats init_package December 19, 2018 12:52:39 PM CST December 19, 2018 12:52:42 PM CST Success
updating the Database version December 19, 2018 12:52:42 PM CST December 19, 2018 12:52:43 PM CST Success
users tablespace creation December 19, 2018 12:52:43 PM CST December 19, 2018 12:52:45 PM CST Success
2.9 安装完成后检查
1)查看集群
[root@oak1 ~]# su - grid
[grid@oak1 ~]$ crs_stat -t -v
Name Type R/RA F/FT Target State Host
----------------------------------------------------------------------
ora....SM.lsnr ora....er.type 0/5 0/ ONLINE ONLINE oak1
ora....SM.lsnr ora....er.type 0/5 0/ ONLINE ONLINE oak1
ora....RE.advm ora....me.type 0/5 0/ ONLINE ONLINE oak1
ora.DATA.dg ora....up.type 0/5 0/ ONLINE ONLINE oak1
ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE oak1
ora....AF.lsnr ora....er.type 0/5 0/ OFFLINE OFFLINE
ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE oak2
ora.MGMTLSNR ora....nr.type 0/0 0/0 ONLINE ONLINE oak2
ora.RECO.dg ora....up.type 0/5 0/ ONLINE ONLINE oak1
ora.REDO.dg ora....up.type 0/5 0/ ONLINE ONLINE oak1
ora.asm ora.asm.type 0/5 0/0 ONLINE ONLINE oak1
ora.chad ora.chad.type 0/5 0/ ONLINE ONLINE oak1
ora.cvu ora.cvu.type 0/5 0/0 ONLINE ONLINE oak2
ora....re.acfs ora.acfs.type 0/5 0/ ONLINE ONLINE oak1
ora.orcl.db ora....se.type 0/2 0/1 ONLINE ONLINE oak1
ora.mgmtdb ora....db.type 0/2 0/1 ONLINE ONLINE oak2
ora....network ora....rk.type 0/5 0/ ONLINE ONLINE oak1
ora....SM1.asm application 0/5 0/0 ONLINE ONLINE oak1
ora....31.lsnr application 0/5 0/0 ONLINE ONLINE oak1
ora.oak1.ons application 0/3 0/0 ONLINE ONLINE oak1
ora.oak1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE oak1
ora....SM2.asm application 0/5 0/0 ONLINE ONLINE oak2
ora....32.lsnr application 0/5 0/0 ONLINE ONLINE oak2
ora.oak2.ons application 0/3 0/0 ONLINE ONLINE oak2
ora.oak2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE oak2
ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE oak1
ora.proxy_advm ora....vm.type 0/5 0/ ONLINE ONLINE oak1
ora.qosmserver ora....er.type 0/1 0/2 ONLINE ONLINE oak2
ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE oak2
2)查看磁盘
[grid@oak1 ~]$ asmcmd
ASMCMD> lsdg
State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED HIGH N 512 512 4096 4194304 12209920 12079688 4883968 2398573 0 Y DATA/
MOUNTED HIGH N 512 512 4096 4194304 3051520 3050452 1220608 609948 0 N RECO/
MOUNTED HIGH N 512 512 4096 4194304 3051520 3001744 762880 746288 0 N REDO/
ASMCMD>
3) 查看实例等(略)
3.附件 ODA X7-2 HA OLTP 资源类型表
4. Ref
更多参考详见:Database Appliance X7-2 Deployment and User’s Guide