1、安装环境：
基于三台VMWARE虚拟机，概况如下：
主机名 | IP | 基础配置 |
node1 | 192.168.20.20 | 1 cpu + 1G |
node2 | 192.168.20.21 | |
node3 | 192.168.20.22 | |
Virtual-IP | 192.168.20.23 |
2、安装前的准备：
（1）禁用SSH进行DNS解析
# sed -i "s/#UseDNS yes/UseDNS no/" /etc/ssh/sshd_config
# systemctl restart sshd
（2）关闭和禁用防火
# systemctl stop firewalld
# systemctl disable firewalld
（3）关闭和禁用selinux
# sed -i 's/^SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
# setenforce 0
3、配置主机时间同步：
# yum install -y ntpdate
# ntpdate time.windows.com && hwclock -w
4、上传安装包
etcd、postgresql、patroni、以及相关依赖包；
序号 | 安装包文件名 | 文件大小 |
1 | etcd-3.3.11-2.el7.centos.x86_64.rpm | 10,491,692 |
2 | postgresql-12.0.tar.gz | 26,393,892 |
3 | python3-cdiff-1.0-1.rhel7.noarch.rpm | 25,840 |
4 | python36-psutil-5.6.7-1.el7.x86_64.rpm | 404,713 |
5 | libpq5-14.1-42PGDG.rhel7.x86_64.rpm | 449,800 |
6 | python3-psycopg2-2.8.6-1.rhel7.x86_64.rpm | 177,120 |
7 | python3-ydiff-1.2-10.rhel7.noarch.rpm | 26,180 |
8 | libyaml-0.1.4-11.el7_0.x86_64.rpm | 56,068 |
9 | python36-PyYAML-3.13-1.el7.x86_64.rpm | 153,058 |
10 | python36-six-1.14.0-3.el7.noarch.rpm | 34,779 |
11 | python36-prettytable-0.7.2-19.el7.noarch.rpm | 41,763 |
12 | python36-dateutil-2.4.2-5.el7.noarch.rpm | 82,992 |
13 | libtommath-0.42.0-6.el7.x86_64.rpm | 36,396 |
14 | libtommath-devel-0.42.0-6.el7.x86_64.rpm | 14,876 |
15 | libtomcrypt-1.17-26.el7.x86_64.rpm | 229,484 |
16 | python36-crypto-2.6.1-16.el7.x86_64.rpm | 498,301 |
17 | python3-etcd-0.4.5-20.rhel7.noarch.rpm | 79,168 |
18 | patroni-etcd-2.1.2-1.rhel7.x86_64.rpm | 3,964 |
19 | python36-dns-1.16.0-1.el7.noarch.rpm | 253,455 |
20 | python36-certifi-2018.10.15-5.el7.noarch.rpm | 12,878 |
21 | python36-pysocks-1.6.8-7.el7.noarch.rpm | 31,230 |
22 | python36-urllib3-1.25.6-2.el7.noarch.rpm | 182,627 |
23 | python36-click-6.7-8.el7.noarch.rpm | 129,712 |
24 | patroni-2.1.2-1.rhel7.x86_64.rpm | 880,040 |
5、安装etcd
# rpm -ivh etcd-3.3.11-2.el7.centos.x86_64.rpm
6、配置etcd
# vi /etc/etcd/etcd.conf
节点 | 配置文件内容 |
node1 | ETCD_NAME="etcd01" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_CLIENT_URLS="http://192.168.20.20:2379,http://127.0.0.1:2379" ETCD_ADVERTISE_CLIENT_URLS="http://192.168.20.20:2379" ETCD_LISTEN_PEER_URLS="http://192.168.20.20:2380" ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.20.20:2380" ETCD_INITIAL_CLUSTER="etcd01=http://192.168.20.20:2380,etcd02=http://192.168.20.21:2380,etcd03=http://192.168.20.22:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" |
node2 | ETCD_NAME="etcd02" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_CLIENT_URLS="http://192.168.20.21:2379,http://127.0.0.1:2379" ETCD_ADVERTISE_CLIENT_URLS="http://192.168.20.21:2379" ETCD_LISTEN_PEER_URLS="http://192.168.20.21:2380" ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.20.21:2380" ETCD_INITIAL_CLUSTER="etcd01=http://192.168.20.20:2380,etcd02=http://192.168.20.21:2380,etcd03=http://192.168.20.22:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" |
node3 | ETCD_NAME="etcd03" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_CLIENT_URLS="http://192.168.20.22:2379,http://127.0.0.1:2379" ETCD_ADVERTISE_CLIENT_URLS="http://192.168.20.22:2379" ETCD_LISTEN_PEER_URLS="http://192.168.20.22:2380" ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.20.22:2380" ETCD_INITIAL_CLUSTER="etcd01=http://192.168.20.20:2380,etcd02=http://192.168.20.21:2380,etcd03=http://192.168.20.22:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" |
7、启动etcd
# systemctl start etcd
8、安装PG
（1）安装依赖包
# yum install -y perl-ExtUtils-Embed readline-devel zlib-devel libxml2-devel libxslt-devel gcc make libpq5-devel python3-devel
（2）创建postgres用户
# useradd postgres
# echo postgres|passwd --stdin postgres
（3）设置postgres用户环境变量
# su - postgres
$ vi ~/.bash_profile
export PGHOME=/opt/pgsql export PGDATA=/data/pgdata export PATH=$PGHOME/bin:$PATH:. export LD_LIBRARY_PATH=$PGHOME/lib:$LD_LIBRARY_PATH export PGPORT=5432 export PGUSER=postgres export PGDATABASE=postgres |
$ source ~/.bash_profile
（4）设置postgres用户的sudo权限：
# vi /etc/sudoers (在第101行加入）
postgres ALL=(root) NOPASSWD: /usr/sbin/ip
（5）创建PG的安装目录和数据库目录：
# mkdir -p /opt/pgsql
# mkdir -p /data/pgdata
（6）解压PG的安装包：
# tar -zxvf postgresql-12.0.tar.gz -C /opt/pgsql/
说明：此处可以选用PG（ https://www.postgresql.org/ftp/source/ ）
但更推荐使用国产的 IvorySQL，它是与PG完全兼容，而且性能更好的国产开源数据库。其官网信息详见 https://www.ivorysql.org/zh-cn/
关于IvorySQL的编译安装，可参考 https://www.modb.pro/db/431542
（7）设置PG相关目录的属主和权限：
# chown -R postgres:postgres /opt/pgsql
# chown -R postgres:postgres /data/pgdata
# chmod 700 /data/pgdata
（8）编译安装PG：
# su - postgres
$ cd /opt/pgsql/postgresql-12.0
$ ./configure --prefix=/opt/pgsql/ --with-perl --with-libxml --with-libxslt
$ make && make install
9、安装Patroni
rpm -ivh python3-cdiff-1.0-1.rhel7.noarch.rpm
rpm -ivh python36-psutil-5.6.7-1.el7.x86_64.rpm
rpm -ivh libpq5-14.1-42PGDG.rhel7.x86_64.rpm
rpm -ivh python3-psycopg2-2.8.6-1.rhel7.x86_64.rpm
rpm -ivh python3-ydiff-1.2-10.rhel7.noarch.rpm
rpm -ivh libyaml-0.1.4-11.el7_0.x86_64.rpm
rpm -ivh python36-PyYAML-3.13-1.el7.x86_64.rpm
rpm -ivh python36-six-1.14.0-3.el7.noarch.rpm
rpm -ivh python36-prettytable-0.7.2-19.el7.noarch.rpm
rpm -ivh python36-dateutil-2.4.2-5.el7.noarch.rpm
rpm -ivh libtommath-0.42.0-6.el7.x86_64.rpm
rpm -ivh libtommath-devel-0.42.0-6.el7.x86_64.rpm
rpm -ivh libtomcrypt-1.17-26.el7.x86_64.rpm
rpm -ivh python36-crypto-2.6.1-16.el7.x86_64.rpm
rpm -ivh python3-etcd-0.4.5-20.rhel7.noarch.rpm
rpm -ivh patroni-etcd-2.1.2-1.rhel7.x86_64.rpm
rpm -ivh python36-dns-1.16.0-1.el7.noarch.rpm
rpm -ivh python36-certifi-2018.10.15-5.el7.noarch.rpm
rpm -ivh python36-pysocks-1.6.8-7.el7.noarch.rpm
rpm -ivh python36-urllib3-1.25.6-2.el7.noarch.rpm
rpm -ivh python36-click-6.7-8.el7.noarch.rpm
rpm -ivh patroni-2.1.2-1.rhel7.x86_64.rpm
10、配置patroni系统服务
# vi /usr/lib/systemd/system/patroni.service
[Unit] Description=Runners to orchestrate a high-availability PostgreSQL After=syslog.target network.target [Service] Type=simple User=postgres Group=postgres EnvironmentFile=-/etc/patroni/patroni_env.conf ExecStartPre=-/usr/bin/sudo /sbin/modprobe softdog ExecStartPre=-/usr/bin/sudo /bin/chown postgres /dev/watchdog ExecStart=/usr/bin/env patroni /etc/patroni/patroni.yml ExecReload=/bin/kill -s HUP $MAINPID KillMode=process TimeoutSec=30 Restart=on-failure [Install] WantedBy=multi-user.target |
11、重新加载systemd服务
# systemctl daemon-reload
12、安装watchdog
使用watchdog为防止出现脑裂
（1）安装软件，linux内置功能
# yum install -y watchdog
（2）初始化watchdog字符设备
# modprobe softdog
（3）修改/dev/watchdog设备权限
# chmod 666 /dev/watchdog
（4）启动watchdog服务
# systemctl start watchdog
# systemctl enable watchdog
13、配置patroni
# mkdir -p /etc/patroni
# chown -R postgres:postgres /etc/patroni
# su - postgres
$ vi /etc/patroni/patroni.yml
scope: pgsql namespace: /service/ name: pg20 restapi: listen: 0.0.0.0:8008 connect_address: 192.168.20.20:8008 etcd3: hosts: 192.168.20.20:2379,192.168.20.21:2379,192.168.20.22:2379 bootstrap: dcs: ttl: 30 loop_wait: 10 retry_timeout: 300 maximum_lag_on_failover: 1048576 master_start_timeout: 300 synchronous_mode: true synchronous_mode_strict: true postgresql: use_pg_rewind: true use_slots: true parameters: listen_addresses: "0.0.0.0" port: 5432 wal_level: replica hot_standby: "on" wal_keep_segments: 100 max_wal_senders: 10 max_replication_slots: 10 wal_log_hints: "on" archive_mode: "on" archive_timeout: 1800s archive_command: mkdir -p /data/pgdata/wal_archive && test ! -f /data/pgdata/wal_archive/%f && cp %p /data/pgdata/wal_archive/%f recovery_conf: restore_command: cp /data/pgdata/wal_archive/%f %p initdb: - encoding: UTF8 - locale: C - lc-ctype: zh_CN.UTF-8 - data-checksums pg_hba: - host replication postgres 192.168.20.0/24 md5 - host all all 0.0.0.0/0 md5 postgresql: listen: 0.0.0.0:5432 connect_address: 192.168.20.20:5432 data_dir: /data/pgdata bin_dir: /opt/pgsql/bin authentication: replication: username: postgres password: "postgres" superuser: username: postgres password: "postgres" rewind: username: postgres password: "postgres" basebackup: max-rate: 100M checkpoint: fast callbacks: on_start: /bin/bash /etc/patroni/patroni_callback.sh on_stop: /bin/bash /etc/patroni/patroni_callback.sh on_role_change: /bin/bash /etc/patroni/patroni_callback.sh watchdog: mode: automatic device: /dev/watchdog safety_margin: 5 tags: nofailover: false noloadbalance: false clonefrom: false nosync: false |
注意：
1、文件格式是yaml文本，不支持使用tab，否则会报错
2、name：后的值，对于node1~3，分别设置pg20、pg21、pg22
3、restapi.connect_address 设置为各自节点IP
4、postgresql.connect_address 设置为各自节点IP
创建patroni的callbak脚本：
$ vi /etc/patroni/patroni_callback.sh
#!/bin/bash readonly OPERATION=$1 readonly ROLE=$2 readonly SCOPE=$3 VIP='192.168.20.23' PREFIX='24' BRD='192.168.20.255' INF='ens32' function usage() { echo "Usage: $0 <on_start|on_stop|on_role_change> <ROLE> <SCOPE>"; exit 1; } echo "$(date "+%Y-%m-%d %H:%M:%S %z") This is patroni callback $OPERATION $ROLE $SCOPE" case $OPERATION in on_stop) sudo ip addr del ${VIP}/${PREFIX} dev ${INF} label ${INF}:1 echo "$(date "+%Y-%m-%d %H:%M:%S %z") VIP ${VIP} removed" ;; on_start | on_restart | on_role_change) if [[ $ROLE == 'master' || $ROLE == 'standby_leader' ]]; then sudo ip addr add ${VIP}/${PREFIX} brd ${BRD} dev ${INF} label ${INF}:1 sudo arping -q -A -c 1 -I ${INF} ${VIP} echo "$(date "+%Y-%m-%d %H:%M:%S %z") VIP ${VIP} added" else sudo ip addr del ${VIP}/${PREFIX} dev ${INF} label ${INF}:1 echo "$(date "+%Y-%m-%d %H:%M:%S %z") VIP ${VIP} removed" fi ;; *) usage ;; esac |
$ chmod u+x /etc/patroni/patroni_callback.sh
14、启动patroni服务：
（1）先确认etcd是否已经启动，并确认状态健康：
# systemctl status etcd
# export ETCDCTL_API=3
# etcdctl endpoint health --endpoints='192.168.20.20:2379,192.168.20.21:2379,192.168.20.22:2379'
（2）启动patroni，会自动初始化PG数据库，并拉起PG服务：
# systemctl start patroni
# systemctl status patroni
15、查看patroni集群状态：
# patronictl -c /etc/patroni/patroni.yml list
