Oracle 11g RAC部署

OS：Oracle Linux 6.5_64 Mini Desktop
Oracle版本：112040_Linux-x86-64

1.概述和说明
1.1概述

1.2软件获取

2.准备工作
2.1介质分配

备注：请提前规划好IP地址、主机名以及硬件等。
相关的rpm包放在/root/rpm/下 和 oracle安装包都放在/root下

2.2新建虚拟机

打开VMware主页
点击”创建新的虚拟机”
创建主机要求：
节点1：Linux 64位，rac1，2核心CPU，8G内存，桥接网卡/NAT网卡，40G硬盘单个文件。
节点2：Linux 64位，rac2，2核心CPU，8G内存，桥接网卡/NAT网卡，40G硬盘单个文件。
共享存储：独立永久磁盘4块，可在安装完系统并关机后添加
        0.5G+0.5G GRIDDG磁盘组，存放OCR和Voting Disk
        3G 配置DATA磁盘组，存放数据库
        3G 配置FLASH磁盘组，用于闪回区
备注：磁盘高级里面，设置虚拟节点0.5G(SCSI1:0) 0.5G(SCSI1:1) 3G(SCSI 2:0) 3G(SCSI 2:1)，2台节点必须设置一致，否则会错乱。
截图

修改虚拟机文件：rac1.vmx和rac2.vmx，使用记事本打开，添加如下内容：

disk.locking = "FALSE"
diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize = "0"
diskLib.DataCacheMinReadAheadSize = "0"
diskLib.dataCachePageSize = "4096"
diskLib.maxUnsyncedWrites = "0"
scsi2.sharedBus = "virtual"
scsi1:0.deviceType = "disk"
scsi1:1.deviceType = "disk"
scsi2:0.deviceType = "disk"
scsi2:1.deviceType = "disk"

最后启动虚拟机

2.3系统安装

系统安装参考Oracle Linux 6系统安装

2.4 本地yum源部署

#rac1 和rac2上操作
#移动本地源public-yum-ol6.repo
[root@rac1 yum.repos.d]# mv public-yum-ol6.repo public-yum-ol6.repo.bak

#创建本地源，上传linux oracle iso包至/mnt下，并挂载至/media
[root@rac1 yum.repos.d]# mount –o loop mnt/OracleLinux-R6-U5-Server-x86_64-dvd.iso media
[root@rac1 yum.repos.d]# vi localhost.repo
[localhost]
name=localhost yum
baseurl=file:///media/
gpgcheck=0
enabled=1

#生成缓存
[root@rac1 yum.repos.d]# yum makecache

3.配置hosts

#用root用户分别在rac1和rac2上关闭防火墙
service iptables stop
chkconfig iptables off

#root登录rac1和rac2分别修改hosts，内容如下
cat etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1       localhost
::1             localhost6.localdomain6 localhost6
# rac1
192.168.100.99  rac1.localdomain        rac1
192.168.100.100 rac1-vip.localdomain    rac1-vip
1.1.1.2         rac1-priv.localdomain   rac1-priv
# rac2
192.168.100.98  rac2.localdomain        rac2
192.168.100.97  rac2-vip.localdomain    rac2-vip
1.1.1.3         rac2-priv.localdomain   rac2-priv
# scan-ip
192.168.100.102 scan-cluster.localdomain        scan-cluster

4.执行脚本

环境变量分配：

root用户分别在rac1和rac2执行以下脚本：

也可在此处下载所有脚本

1.sh:
#!/bin/bash
#Purpose:Create 6 groups named 'oinstall','dba','asmadmin','asmdba','asmoper','oper', plus 2 users named 'oracle','grid'.
#Also setting the Environment
#variable for oracle user.
#variable for grid user.
#Usage:Log on as the superuser('root'),and then execute the command:#./1preusers.sh
#Author:Asher Huang
 
echo "Now create 6 groups named 'oinstall','dba','asmadmin','asmdba','asmoper','oper'"
echo "Plus 2 users named 'oracle','grid',Also setting the Environment"

 
groupadd -g 1000 oinstall
groupadd -g 1200 asmadmin
groupadd -g 1201 asmdba
groupadd -g 1202 asmoper
useradd -u 1100 -g oinstall -G asmadmin,asmdba,asmoper -d home/grid -s bin/bash -c "grid Infrastructure Owner" grid
echo "grid" | passwd --stdin grid
 
echo 'export PS1="`/bin/hostname -s`-> "'>> home/grid/.bash_profile
echo "export TMP=/tmp">> home/grid/.bash_profile
echo 'export TMPDIR=$TMP'>>/home/grid/.bash_profile
echo "export ORACLE_SID=+ASM1">> home/grid/.bash_profile
echo "export ORACLE_BASE=/main/app/grid">> home/grid/.bash_profile
echo "export ORACLE_HOME=/main/app/11.2.0/grid">> home/grid/.bash_profile
echo "export ORACLE_TERM=xterm">> home/grid/.bash_profile
echo "export NLS_DATE_FORMAT='yyyy/mm/dd hh24:mi:ss'" >> home/grid/.bash_profile
echo 'export TNS_ADMIN=$ORACLE_HOME/network/admin'  >> home/grid/.bash_profile
echo 'export PATH=/usr/sbin:$PATH'>> home/grid/.bash_profile
echo 'export PATH=$ORACLE_HOME/bin:$PATH'>> home/grid/.bash_profile
echo 'export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib'>> home/grid/.bash_profile
echo 'export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib'>> home/grid/.bash_profile
echo "export EDITOR=vi" >> home/grid/.bash_profile
echo "export LANG=en_US" >> home/grid/.bash_profile
echo "export NLS_LANG=american_america.AL32UTF8" >> home/grid/.bash_profile
echo "umask 022">> home/grid/.bash_profile
	 
groupadd -g 1300 dba
groupadd -g 1301 oper
useradd -u 1101 -g oinstall -G dba,oper,asmdba -d home/oracle -s bin/bash -c "Oracle Software Owner" oracle
echo "oracle" | passwd --stdin oracle
 
echo 'export PS1="`/bin/hostname -s`-> "'>> home/oracle/.bash_profile
echo "export TMP=/tmp">> home/oracle/.bash_profile
echo 'export TMPDIR=$TMP'>>/home/oracle/.bash_profile
echo "export ORACLE_HOSTNAME=node1.localdomain">> home/oracle/.bash_profile
echo "export ORACLE_SID=devdb1">> home/oracle/.bash_profile
echo "export ORACLE_BASE=/main/app/oracle">> home/oracle/.bash_profile
echo 'export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1'>> home/oracle/.bash_profile
echo "export ORACLE_UNQNAME=devdb">> home/oracle/.bash_profile
echo 'export TNS_ADMIN=$ORACLE_HOME/network/admin'  >> home/oracle/.bash_profile
echo "export ORACLE_TERM=xterm">> home/oracle/.bash_profile
echo 'export PATH=/usr/sbin:$PATH'>> home/oracle/.bash_profile
echo 'export PATH=$ORACLE_HOME/bin:$PATH'>> home/oracle/.bash_profile
echo 'export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib'>> home/oracle/.bash_profile
echo 'export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib'>> home/oracle/.bash_profile
echo "export EDITOR=vi" >> home/oracle/.bash_profile
echo "export LANG=en_US" >> home/oracle/.bash_profile
echo "export NLS_LANG=american_america.AL32UTF8" >> home/oracle/.bash_profile
echo "export NLS_DATE_FORMAT='yyyy/mm/dd hh24:mi:ss'" >> home/oracle/.bash_profile
echo "umask 022">> home/oracle/.bash_profile
 
echo "The Groups and users has been created"
echo "The Environment for grid,oracle also has been set successfully"

2.sh:
#!/bin/bash
#Purpose:Create the necessary directory for oracle,grid users and change the authention to oracle,grid users.
#Usage:Log on as the superuser('root'),and then execute the command:#./2predir.sh
#Author:Asher Huang
echo "Now create the necessary directory for oracle,grid users and change the authention to oracle,grid users..."
mkdir -p main/app/grid
mkdir -p main/app/11.2.0/grid
mkdir -p main/app/oracle
chown -R oracle:oinstall main
chown -R grid:oinstall main/app/grid
chown -R grid:oinstall main/app/11.2.0
chmod -R 775 main
echo "The necessary directory for oracle,grid users and change the authention to oracle,grid users has been finished"



3.sh:
#!/bin/bash
#Purpose:Change the etc/security/limits.conf.
#Usage:Log on as the superuser('root'),and then execute the command:#./3prelimits.sh
#Author:Asher Huang
 
echo "Now modify the etc/security/limits.conf,but backup it named etc/security/limits.conf.bak before"
cp etc/security/limits.conf etc/security/limits.conf.bak
echo "oracle soft nproc 2047" >>/etc/security/limits.conf
echo "oracle hard nproc 16384" >>/etc/security/limits.conf
echo "oracle soft nofile 1024" >>/etc/security/limits.conf
echo "oracle hard nofile 65536" >>/etc/security/limits.conf
echo "grid soft nproc 2047" >>/etc/security/limits.conf
echo "grid hard nproc 16384" >>/etc/security/limits.conf
echo "grid soft nofile 1024" >>/etc/security/limits.conf
echo "grid hard nofile 65536" >>/etc/security/limits.conf
echo "Modifing the etc/security/limits.conf has been succeed."

4.sh:
#!/bin/bash
#Purpose:Modify the etc/pam.d/login.
#Usage:Log on as the superuser('root'),and then execute the command:#./4prelimits.sh
#Author:Asher Huang
echo "Now modify the etc/pam.d/login,but with a backup named etc/pam.d/login.bak"
cp etc/pam.d/login etc/pam.d/login.bak	 
echo "session required lib/security/pam_limits.so" >>/etc/pam.d/login
echo "session required pam_limits.so" >>/etc/pam.d/login
echo "Modifing the etc/pam.d/login has been succeed."

5.sh:
#!/bin/bash
#Purpose:Modify the etc/profile.
#Usage:Log on as the superuser('root'),and then execute the command:#./5preprofile.sh
#Author:Asher Huang
echo "Now modify the  etc/profile,but with a backup named  etc/profile.bak"
cp etc/profile etc/profile.bak
echo 'if [ $USER = "oracle" ]||[ $USER = "grid" ]; then' >>  etc/profile
echo 'if [ $SHELL = "/bin/ksh" ]; then' >> etc/profile
echo 'ulimit -p 16384' >> etc/profile
echo 'ulimit -n 65536' >> etc/profile
echo 'else' >> etc/profile
echo 'ulimit -u 16384 -n 65536' >> etc/profile
echo 'fi' >> etc/profile
echo 'fi' >> etc/profile
echo "Modifing the etc/profile has been succeed."

6.sh:
#!/bin/bash
#Purpose:Modify the etc/sysctl.conf.
#Usage:Log on as the superuser('root'),and then execute the command:#./6presysctl.sh
#Author:Asher Huang
echo "Now modify the etc/sysctl.conf,but with a backup named etc/sysctl.bak"
cp etc/sysctl.conf etc/sysctl.conf.bak
echo "fs.aio-max-nr = 1048576" >> etc/sysctl.conf
echo "fs.file-max = 6815744" >> etc/sysctl.conf
echo "kernel.shmall = 2097152" >> etc/sysctl.conf
echo "kernel.shmmax = 4183418880" >> etc/sysctl.conf
echo "kernel.shmmni = 4096" >> etc/sysctl.conf
echo "kernel.sem = 250 32000 100 128" >> etc/sysctl.conf
echo "net.ipv4.ip_local_port_range = 9000 65500" >> etc/sysctl.conf
echo "net.core.rmem_default = 262144" >> etc/sysctl.conf
echo "net.core.rmem_max = 4194304" >> etc/sysctl.conf
echo "net.core.wmem_default = 262144" >> etc/sysctl.conf
echo "net.core.wmem_max = 1048586" >> etc/sysctl.conf
echo "net.ipv4.tcp_wmem = 262144 262144 262144" >> etc/sysctl.conf
echo "net.ipv4.tcp_rmem = 4194304 4194304 4194304" >> etc/sysctl.conf
echo "Modifing the etc/sysctl.conf has been succeed."
echo "Now make the changes take effect....."
sysctl -p

root用户修改rac1环境

[root@rac1 ~]# vi home/oracle/.bash_profile
把        export ORACLE_HOSTNAME=node1.localdomain
改为      export ORACLE_HOSTNAME=rac1.localdomain

root用户修改rac2环境

[root@rac2 ~]# vi home/oracle/.bash_profile
把        export ORACLE_HOSTNAME=node1.localdomain
改为      export ORACLE_HOSTNAME=rac2.localdomain
把        export ORACLE_SID=devdb1
改为      export ORACLE_SID=devdb2
[root@rac2 ~]# vi home/grid/.bash_profile
把        export ORACLE_SID=+ASM1
改为      export ORACLE_SID=+ASM2

5.配置SSH对等性

5.1停止NTP

#停止NTP，rac1执行，root用户
[root@rac1 ~]# service ntpd stop
[root@rac1 ~]# chkconfig ntpd off
[root@rac1 ~]# cp etc/ntp.conf etc/ntp.conf.bak
[root@rac1 ~]# rm -rf etc/ntp.conf

#停止NTP，rac2执行，root用户
[root@rac2 ~]# service ntpd stop
[root@rac2 ~]# chkconfig ntpd off
[root@rac2 ~]# cp etc/ntp.conf etc/ntp.conf.bak
[root@rac2 ~]# rm -rf etc/ntp.conf

5.2配置SSH对等

为2台节点的oracle和grid用户分别设置SSH无密钥登录
使用root用户分别在登录rac1和rac2执行

oracle用户设置：

[root@rac1 ~]#su - oracle
rac1->mkdir ~/.ssh
rac1->chmod 700 ~/.ssh
rac1->ssh-keygen -t rsa                            ----一直确定即可
rac1->ssh-keygen -t dsa                            ----一直确定即可

[root@rac2 ~]#su - oracle
rac2->mkdir ~/.ssh
rac2->chmod 700 ~/.ssh
rac2->ssh-keygen -t rsa                            ----一直确定即可
rac2->ssh-keygen -t dsa                            ----一直确定即可

#在oracle用户下继续执行
rac1->cat ~/.ssh/id_rsa.pub >>~/.ssh/authorized_keys
rac1->cat ~/.ssh/id_dsa.pub >>~/.ssh/authorized_keys
rac1->ssh rac2 cat ~/.ssh/id_rsa.pub >>~/.ssh/authorized_keys
rac1->ssh rac2 cat ~/.ssh/id_dsa.pub >>~/.ssh/authorized_keys

rac2->cat ~/.ssh/id_rsa.pub >>~/.ssh/authorized_keys
rac2->cat ~/.ssh/id_dsa.pub >>~/.ssh/authorized_keys
rac2->ssh rac1 cat ~/.ssh/id_rsa.pub >>~/.ssh/authorized_keys
rac2->ssh rac1 cat ~/.ssh/id_dsa.pub >>~/.ssh/authorized_keys

下一步是消除ssh连接时的提示，输入yes

rac1-> ssh rac1 date
rac1-> ssh rac2 date
rac1-> ssh rac1-priv date
rac1-> ssh rac2-priv date
rac1-> ssh rac1.localdomain date
rac1-> ssh rac2.localdomain date
rac1-> ssh rac1-priv.localdomain date
rac1-> ssh rac2-priv.localdomain date

rac2-> ssh rac1 date
rac2-> ssh rac2 date
rac2-> ssh rac1-priv date
rac2-> ssh rac2-priv date
rac2-> ssh rac1.localdomain date
rac2-> ssh rac2.localdomain date
rac2-> ssh rac1-priv.localdomain date
rac2-> ssh rac2-priv.localdomain date

grid用户设置：

[root@rac1 ~]#su – grid
rac1->mkdir ~/.ssh
rac1->chmod 700 ~/.ssh
rac1->ssh-keygen -t rsa                            ----一直确定即可
rac1->ssh-keygen -t dsa                            ----一直确定即可

[root@rac2 ~]#su – grid
rac2->mkdir ~/.ssh
rac2->chmod 700 ~/.ssh
rac2->ssh-keygen -t rsa                            ----一直确定即可
rac2->ssh-keygen -t dsa                            ----一直确定即可

#在grid用户下继续执行
rac1->cat ~/.ssh/id_rsa.pub >>~/.ssh/authorized_keys
rac1->cat ~/.ssh/id_dsa.pub >>~/.ssh/authorized_keys
rac1->ssh rac2 cat ~/.ssh/id_rsa.pub >>~/.ssh/authorized_keys
rac1->ssh rac2 cat ~/.ssh/id_dsa.pub >>~/.ssh/authorized_keys

rac2->cat ~/.ssh/id_rsa.pub >>~/.ssh/authorized_keys
rac2->cat ~/.ssh/id_dsa.pub >>~/.ssh/authorized_keys
rac2->ssh rac1 cat ~/.ssh/id_rsa.pub >>~/.ssh/authorized_keys
rac2->ssh rac1 cat ~/.ssh/id_dsa.pub >>~/.ssh/authorized_keys

下一步是消除ssh连接时的提示，输入yes

rac1-> ssh rac1 date
rac1-> ssh rac2 date
rac1-> ssh rac1-priv date
rac1-> ssh rac2-priv date
rac1-> ssh rac1.localdomain date
rac1-> ssh rac2.localdomain date
rac1-> ssh rac1-priv.localdomain date
rac1-> ssh rac2-priv.localdomain date

rac2-> ssh rac1 date
rac2-> ssh rac2 date
rac2-> ssh rac1-priv date
rac2-> ssh rac2-priv date
rac2-> ssh rac1.localdomain date
rac2-> ssh rac2.localdomain date
rac2-> ssh rac1-priv.localdomain date
rac2-> ssh rac2-priv.localdomain date

6.配置ASM磁盘

6.1格式化磁盘

#在一台节点格式化磁盘即可。
#root用户进入rac1，使用fdisk命令格式化磁盘
[root@rac1 ~]# fdisk dev/sdb

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-500, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-500, default 500):
Using default value 500

Command (m for help): w

The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

#使用相同方法格式化其他3块磁盘

6.2安装ASM服务

软件包下载
oracleasm-support-2.1.8-1.el5.x86_64.rpm

kmod-oracleasm

oracleasmlib-2.0.4-1.el5.x86_64.rpm

kmod-oracleasm-2.0.8-5.el6_7.x86_64.rpm

kernel-2.6.32-573.8.1.el6.x86_64.rpm

dracut-kernel-004-388.0.1.el6.noarch.rpm

dracut-004-388.0.1.el6.noarch.rpm

#注意：选择的软件包必须与操作系统、内核版本一致。
#root用户分别在rac1和rac2上安装ASM软件包，软件包按照顺序安装。
rpm –ivh oracleasm-support-2.1.8-1.el5.x86_64.rpm
yum install –y kmod-oracleasm
rpm –ivh oracleasmlib-2.0.4-1.el5.x86_64.rpm

#在安装kmod-oracleasm时因网络问题会很慢，下载包后安装
yum localinstall  kmod-oracleasm-2.0.8-5.el6_7.x86_64.rpm kernel-2.6.32-573.8.1.el6.x86_64.rpm dracut-kernel-004-388.0.1.el6.noarch.rpm dracut-004-388.0.1.el6.noarch.rpm

6.3配置ASM服务

#root用户分别在rac1和rac2上执行。
[root@rac1 ~]#/usr/sbin/oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
[root@rac1 ~]# usr/sbin/oracleasm configure
ORACLEASM_ENABLED=true
ORACLEASM_UID=grid
ORACLEASM_GID=asmadmin
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""

[root@rac1 ~]#/usr/sbin/oracleasm init

6.4创建ASM磁盘

#root用户登录rac1执行以下命令
[root@rac1 ~]# usr/sbin/oracleasm createdisk VOL1 dev/sdb1
Writing disk header: done
Instantiating disk: done
[root@rac1 ~]# usr/sbin/oracleasm createdisk VOL2 dev/sdc1
Writing disk header: done
Instantiating disk: done
[root@rac1 ~]# usr/sbin/oracleasm createdisk VOL3 dev/sdd1
Writing disk header: done
Instantiating disk: done
[root@rac1 ~]# usr/sbin/oracleasm createdisk VOL4 dev/sde1
Writing disk header: done
Instantiating disk: done

#查看结果：
[root@rac1 ~]# usr/sbin/oracleasm listdisks
VOL1
VOL2
VOL3
VOL4

#root用户进入rac2，执行以下命令，以下显示正常扫描到ASM磁盘
[root@rac2 ~]# usr/sbin/oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks…
Scanning system for ASM disks…
Instantiating disk "VOL1"
Instantiating disk "VOL2"
Instantiating disk "VOL3"
Instantiating disk "VOL4"
[root@rac2 ~]# usr/sbin/oracleasm listdisks
VOL1
VOL2
VOL3
VOL4

其他：查看磁盘和物理磁盘的对应关系
[root@rac1 ~]# /usr/sbin/oracleasm querydisk /dev/sd*

7.安装Grid Infrastructure

7.1介质准备

#使用root用户上传安装包到rac1和rac2，路径无要求，比如/root下，并解压
[root@rac1 ~]#yum install –y unzip
[root@rac1 ~]#unzip p13390677_112040_Linux-x86-64_1of7.zip
[root@rac1 ~]#unzip p13390677_112040_Linux-x86-64_2of7.zip
[root@rac1 ~]#unzip p13390677_112040_Linux-x86-64_3of7.zip

#移动解压包到/home/oracle/和/home/grid/目录下，rac1和rac2都需要操作
[root@rac1 ~]#mv database /home/oracle/
[root@rac1 ~]#mv grid /home/grid/

7.2环境检查

软件包下载
下载依赖包(依赖包放置在/home/oracle/rpm下)
compat-libcap1-1.10-1.x86_64.rpm

compat-libstdc++-33-3.2.3-69.el6.x86_64.rpm

gcc-c++-4.4.7-4.el6.x86_64.rpm

ksh-20120801-10.el6.x86_64.rpm

libaio-0.3.107-10.el6.x86_64.rpm

libaio-devel-0.3.107-10.el6.x86_64.rpm

libstdc++-devel-4.4.7-4.el6.x86_64.rpm

sysstat-9.0.4-22.el6.x86_64.rpm

#只需在rac1上执行检查。
[root@rac1 ~]# su - grid
rac1-> cd grid
rac1-> ./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -fixup -verbose
......省略大量输出

#根据输出提示，root用户分别在rac1和rac2上执行以下命令：
[root@rac1]# /tmp/CVU_11.2.0.4.0_grid/runfixup.sh

#根据输出提示，root用户分别在rac1和rac2上安装未安装的rpm软件包：
rpm –ivh sysstat-9.0.4-22.el6.x86_64.rpm
rpm –ivh libstdc++-devel-4.4.7-4.el6.x86_64.rpm
rpm –ivh libaio-0.3.107-10.el6.x86_64.rpm
rpm –ivh libaio-devel-0.3.107-10.el6.x86_64.rpm
rpm –ivh ksh-20120801-10.el6.x86_64.rpm
rpm –ivh gcc-c++-4.4.7-4.el6.x86_64.rpm
rpm –ihv compat-libstdc++-33-3.2.3-69.el6.x86_64.rpm
rpm –ivh compat-libcap1-1.10-1.x86_64.rpm

#最后还有一个oralce自带的rpm包需要提前安装，2台节点都需执行：
cd /home/grid/grid/rpm/
rpm –ivh cvuqdisk-1.0.9-1.rpm
#在安装此rpm包时若提示：
ls: cannot access /usr/sbin/smartctl: No such file or directory 则需要安装smartctl
#解决方法
yum install smartmontools -y

#以上执行完后基本就没Faild了，若有必须提前解决，否则后面会出错。

7.3开始安装Grid

#前面检测没有问题则可以开始Grid的安装。
#使用grid用户进入系统虚拟机桌面，打开终端，执行以下命令：
rac1->cd grid
rac1->./runInstaller

#以上命令会弹出Grid安装界面，根据截图选择即可：

点击add，增加第二个节点：此处密码后面dbca时会用到此处用户组选择按照默认，此处的3个用户组顺序是会变得，所以按照默认，不要修改。此步需要修复，根据提示修改shmmax值此步需要先在rac1上执行脚本后再去rac2上执行执行信息：执行完毕之后点击OK按钮，继续安装
此步忽略此处错误，点击ok。因为没配置DNS服务器而报错，ping下scan ip如果能ping 通这个问题可以忽略到此步，Grid安装完毕

7.4创建磁盘组

#进入grid用户执行asmca命令
[root@rac1 ~]# su – grid
rac1 -> asmca

#弹出磁盘组设置创建界面

点击Create按钮，开始创建点击ok，过一会提示创建成功再次点击Create按钮，创建闪回区同上点击ok，开始创建，过一会提示创建完成提示
最后点击Exit按钮退出磁盘组创建界面。

查看状态：

8.安装Oracle

#切换至oracle用户，开始安装oracle软件
su – oracle
cd database
./runInstaller

#弹出开始安装界面

此问题，忽略即可点击Install按钮，开始安装弹出执行脚本命令：以root用户先在rac1执行，后在rac2执行，执行信息如下：

执行完毕后点击ok按钮，安装结束

9.创建RAC数据库

#以oralce用户登录，执行dbca命令，开始安装数据库
su – oracle
dbca

#弹出创建数据库界面

此处密码，最好复杂此处密码为7.3章节设置的密码此处的FLASH大小根据实际修改，比如存储分配了100G那么就写99999M开始数据库的建立..至此，RAC数据库安装完毕

10.VNC安装设置

#在真实环境中，需要用到VNC连接，而不用直接操作服务器，还需要grid用户和oracle切换。
#安装vnc(rac1  rac2)
yum install tigervnc tigervnc-server

----设置vnc-------grid用户
vi /etc/sysconfig/vncservers
#末尾添加：
VNCSERVERS="1:grid"	
VNCSERVERARGS[1]="-geometry 1024x768"

#切换到grid用户设置vnc密码：
su - grid
vncserver
#修改xstartup文件
vi /home/grid/.vnc/xstartup
把twm & 改为 gnome-session &

#启动vnc
service vncserver restart

#切换oracle用户时需要停止vnc
service vncserver stop

----设置vnc-------oracle用户
vi /etc/sysconfig/vncservers
#末尾添加：
VNCSERVERS="1:oracle"	
VNCSERVERARGS[1]="-geometry 1024x768"
#切换到grid用户设置vnc密码：
su – oracle
vncserver

#修改xstartup文件
vi /home/grid/.vnc/xstartup
把twm & 改为 gnome-session &

#启动vnc
service vncserver restart

11.故障测试

心跳线断开情况：
1.节点rac2会自动关闭oracle的所有服务
2.观察中发现rac2的vip自动漂移到rac1上

心跳线再次连接上情况：
1.节点rac2会自动启动所有oralce服务
2.恢复到故障前的正常状态

备注：测试环境仅在 oracle 11g rac 双节点环境中测试，并且心跳线是直连的，至于10g的未测试，据说是直接重启系统。
官方建议心跳线通过交换机连接。
当然更安全的是通过双网卡绑定。