暂无图片
暂无图片
暂无图片
暂无图片
暂无图片
首页 / 如何在京东公有云环境下部署oracle18c RAC

如何在京东公有云环境下部署oracle18c RAC

北京IT爷们儿 2021-06-11
459

理论篇

安装oracle RAC　主要需要几个条件

  • 私有网络和公共网络隔离（不在同一网段）

  • 共享存储

  • 网络需要支持mulitcast

我们可以利用京东云提供的基础设施解决以上问题

  • 通过创建"创建无预设CIDR私有网络"的vpc进行网络隔离，实现public和private网段隔离

  • 通过创建"多点挂载存储"实现ASM所需的共享存储

  • 通过n2n点对点vpn实现multicast支持


实施篇


网络配置

  • 创建“创建无预设CIDR私有网络”vpc
    控制台=>网络=>私有网络(vpc)=>创建=>选择"创建无预设CIDR私有网络"

  • 创建子网
    分别创建两个不同网段的子网
    10.0.1.0/24
    192.168.1.0/24


主机创建


创建共享磁盘

控制台=>弹性计算=>云硬盘=>创建=>选择"多点挂载"（目前仅华东有此选项）


创建弹性网卡

192.168.1.10 rac1private
192.168.1.20 rac2private


修改hosts文件

vi etc/hosts


#oracle rac public ip
10.0.0.10 rac1
10.0.0.20 rac2
#oracle rac vip
10.0.0.11 rac1vip
10.0.0.21 rac2vip
#oracle rac scanip
10.0.0.30 scan-rac
#oracle rac private ip
192.168.２.10 rac1private
192.168.２.20 rac2private

这里可以利用弹性网卡来绑定多个ip地址来实现public ip和vip


创建用户

https://docs.oracle.com/en/database/oracle/oracle-database/18/cwlin/identifying-an-oracle-software-owner-user-account.html#GUID-0A95F4B1-1045-455D-9897-A23012E4E27F


用户和组：
/usr/sbin/groupadd -g 54321 oinstall
/usr/sbin/groupadd -g 54322 dba
/usr/sbin/groupadd -g 54323 oper
/usr/sbin/groupadd -g 54324 backupdba
/usr/sbin/groupadd -g 54325 dgdba
/usr/sbin/groupadd -g 54326 kmdba
/usr/sbin/groupadd -g 54327 asmdba
/usr/sbin/groupadd -g 54328 asmoper
/usr/sbin/groupadd -g 54329 asmadmin
/usr/sbin/groupadd -g 54330 racdba
/usr/sbin/useradd -u 54321 -g oinstall -G dba,asmdba,backupdba,dgdba,kmdba,racdba,oper oracle
/usr/sbin/useradd -u 54331 -g oinstall -G dba,asmdba,asmoper,asmadmin grid
echo oracle | passwd --stdin oracle
echo oracle | passwd --stdin grid
11、目录：
mkdir -p u01/app/18.3.0/grid
mkdir -p u01/app/grid
mkdir -p u01/app/oracle
chown -R grid:oinstall u01
chown oracle:oinstall u01/app/oracle
chmod -R 775 u01/
12、环境变量
节点1：
--grid
export ORACLE_SID=+ASM1
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/18.3.0/grid
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
#export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
--oracle用户
export ORACLE_SID=db1
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/app/oracle/product/18.3.0/db_1
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
#export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
节点2：
--gird
export ORACLE_SID=+ASM2
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/18.3.0/grid
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
#export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
--oracle用户
export ORACLE_SID=db2
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/app/oracle/product/18.3.0/db_1
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
#export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

  • 验证用户组


$ grep "oinstall" etc/group
oinstall:x:54321:grid,oracle
You can then use the ID command to verify that the Oracle installation owners you intend to use have the Oracle Inventory group as their primary group. For example:
$ id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba), 
54323(oper),54324(backupdba),54325(dgdba),54326(kmdba),54327(asmdba),54330(racdba)
$ id grid
uid=54331(grid) gid=54321(oinstall) groups=54321(oinstall),54322(dba),
54327(asmdba),54328(asmoper),54329(asmadmin),54330(racdba)


操作系统参数

  • vi etc/sysctl.conf
    sysctl -p etc/sysctl.d/*.conf


fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 67108864
kernel.shmmax = 274877906944
kernel.shmmni = 4096
#kernel.sem = 250 32000 100 128
kernel.sem = 5010 641280 5010 128
net.ipv4.ip_local_port_range = 9000 65500
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 16384 4194304
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576

  • 限制文件：vi etc/security/limits.conf


grid soft nproc 2047 
grid hard nproc 16384 
grid soft nofile 1024 
grid hard nofile 65536 
grid soft stack 10240 
oracle soft nproc 2047 
oracle hard nproc 16384 
oracle soft nofile 1024 
oracle hard nofile 65536 
oracle soft stack 10240
* soft memlock 524288000
* hard memlock 524288000

  • vi etc/pam.d/login 文件
    session required pam_limits.so

  • vi etc/fstab
    tmpfs dev/shm tmpfs size=5000m 0 0

  • /etc/sysconfig/network
    NOZEROCONF=yes

  • vi etc/profile


if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
    if [ $SHELL = "/bin/ksh" ]; then
        ulimit -p 16384
        ulimit -n 65536
    else
        ulimit -u 16384 -n 65536
    fi
    umask 022
fi


安装必要包


yum install -y gcc-c++
yum install -y bc
yum install -y binutils
yum install -y compat-libcap1
yum install -y compat-libstdc++-33.i686    
yum install -y compat-libstdc++-33.x86_64
yum install -y glibc-2.17-36.el7.i686
yum install -y glibc-2.17-36.el7.x86_64
yum install -y glibc-devel.i686
yum install -y glibc-devel.x86_64
yum install -y ksh
yum install -y libaio.i686
yum install -y libaio.x86_64
yum install -y libaio-devel.i686
yum install -y libaio-devel.x86_64
yum install -y libX11.i686
yum install -y libX11.x86_64
yum install -y libXau.i686
yum install -y libXau.x86_64
yum install -y libXi.i686
yum install -y libXi.x86_64
yum install -y libXtst.i686
yum install -y libXtst.el7.x86_64
yum install -y libgcc.i686
yum install -y libgcc.x86_64
yum install -y libstdc++.i686
yum install -y libstdc++
yum install -y libstdc++-devel.i686
yum install -y libstdc++-devel.x86_64
yum install -y libxcb.i686
yum install -y libxcb.x86_64
yum install -y make
yum install -y nfs-utils 
yum install -y net-tools
yum install -y python
yum install -y python-configshell
yum install -y python-rtslib
yum install -y python-six
yum install -y smartmontools
yum install -y sysstat
yum install -y targetcli

  • 为保证xwindow正常显示的必要安装


yum groupinstall "X Window System" "Fonts" -y


停止ntp服务


systemctl stop chronyd
systemctl disable chronyd
mv etc/chrony.conf etc/chrony.conf_bak
# sbin/service ntpd stop
# chkconfig ntpd off
# mv /etc/ntp.conf /etc/ntp.conf.org
# rm /var/run/ntpd.pid


准备asm磁盘组（所有节点操作）

  • asmlib方式
    参考文档：https://docs.microsoft.com/zh-tw/azure/virtual-machines/workloads/oracle/configure-oracle-asm

    /usr/sbin/oracleasm configure -i
    Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done

    此处需要reboot

    oracleasm createdisk ocr /dev/vdb1
oracleasm createdisk data /dev/vdc1
oracleasm scandisks
oracleasm listdisks
    dd if=/dev/zero of=/dev/vdb1 bs=1M count=1000

    • 清除磁盘数据

    • 创建asm磁盘组

    • 从 https://www.oracle.com/technetwork/server-storage/linux/asmlib/rhel7-2773795.html 下载rpm

    • 安装
      rpm -Uvh oracleasm-support-2.1.11-2.el7.x86_64.rpm
      yum -y install kmod-oracleasm.x86_64
      rpm -ivh oracleasmlib-2.0.12-1.el7.x86_64.rpm

    • 配置asm


n2n安装及配置

  • 安装

    yum install -y gcc
yum install -y openssl-devel
    git clone https://github.com/ntop/n2n.git
cd n2n
git checkout 2.4-stable
make
make PREFIX=/opt/n2n install

  • 安装完毕后启动supernode服务

    nohup /opt/n2n/sbin/supernode -l 65530 &

  • 在RAC节点安装ntop软件（计划加入vpn网络的节点都需要安装ntop）

    scp -r /opt/n2n 10.0.0.10:/opt
scp -r /opt/n2n 10.0.0.20:/opt
    /opt/n2n/sbin/edge -d edge1 -a 192.168.2.10 -s 255.255.255.0 -c dtstack -k  dtstack -l 192.168.1.40:65530 -E -r
    /opt/n2n/sbin/edge -d edge1 -a 192.168.2.20 -s 255.255.255.0 -c dtstack -k  dtstack -l 192.168.1.40:65530 -E -r

    • 节点2

    • 节点1

  • 测试multicast(非必要，主要是验证multicast无异常)

    ./mcasttest.pl -n rac1,rac2 -i edge1


ssh互信

  • 分别在grid和oracle用户，每个节点上执行。

    mkdir .ssh
chmod 700 ~/.ssh
cd ~/.ssh
    ssh-keygen -t rsa
ssh-keygen -t dsa

    • 生成秘钥

    • 创建ssh目录

  • 在rac1上将密钥添加到授权文件中,grid和oracle用户分别执行

    cat id_dsa.pub >> authorized_keys
cat id_rsa.pub >> authorized_keys
    ssh rac2 cat ~/.ssh/id_dsa.pub >> authorized_keys
ssh rac2 cat ~/.ssh/id_rsa.pub >> authorized_keys
    scp authorized_keys rac2:/home/grid/.ssh/
scp authorized_keys rac2:/home/oracle/.ssh/

    • 将本机的authorized_keys的文件拷贝到另一台主机上

    • 拷贝另一台主机的秘钥

    • 本机的拷贝

  • 修改两个主机的授权密钥的权限

    chmod 600 ~/.ssh/authorized_keys

  • 测试

    ssh-keyscan rac1  >> ~/.ssh/known_hosts
ssh-keyscan rac2  >> ~/.ssh/known_hosts
ssh-keyscan rac1private  >> ~/.ssh/known_hosts
ssh-keyscan rac2private  >> ~/.ssh/known_hosts
    ssh rac1 date
ssh rac2 date
ssh rac1private date
ssh rac2private date

    • 测试

    • 获取服务器公钥


安装前检查


$ORACLE_HOME/runcluvfy.sh stage -pre crsinst -n rac1,rac2 -verbose

  • 错误处理

    tmpfs  /dev/shm tmpfs  size=5000m   0 0

    • PRVE-0421
      修改/etc/fstab


安装GI--grid用户下

  • 解压安装压缩包

    cd /u01/app/12.2.0/grid/grid
unzip /oraclesoftware/linuxx64_12201_grid_home.zip

  • 安装cvudisk RPM包
    root 用户安装
    cvuqdisk存于oracle GI安装介质的cv/rpm目录下

    cd /u01/app/18.3.0/grid
export CVUQDISK_GRP=asmadmin
rpm -ivh cv/rpm/cvuqdisk-1.0.10-1.rpm
scp cv/rpm/cvuqdisk-1.0.10-1.rpm 10.0.0.20:/root
export CVUQDISK_GRP=asmadmin
rpm -ivh /root/cvuqdisk-1.0.10-1.rpm

  • grid用户执行$HOME_ORACLE/gridSetup.sh


安装DB--oracle用户下

  • 解压安装压缩包

    cd /u01/app/oracle/product/18.3.0/db_1
unzip /oraclesoftware/LINUX.X64_180000_db_home.zip

  • oracle用户执行$HOME_ORACLE/runInstaller.sh


查看crs服务状态


crsctl status res -t




oracle
文章转载自北京IT爷们儿，如果涉嫌侵权，请发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。

评论