制作he3db centos8源码编译docker环境指导

原创 cxp 2023-11-02

442

Docker镜像

Pass部私仓：

Docker镜像制作具体流程

（1）下载centos8镜像

sudo docker pull centos:centos8
sudo docker run -itd --name c8 centos:centos8
sudo docker exec -it c8 /bin/bash

（2）进入docker容器环境变更centos8的yum源

mv /etc/yum.repos.d/ /etc/yum.repos.d\_bak && mkdir /etc/yum.repos.d/
curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-vault-8.5.2111.repo
sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo
dnf makecache

（3）创建数据库用户

yum -y update
yum -y install passwd sudo zip unzip tar
yum -y install git

----创建数据库用户

useradd postgres
passwd postgres

----给用户sudo权限（看实际情况给予）

chmod u+w /etc/sudoers
vi /etc/sudoers
chmod u-w /etc/sudoers

（4）切换到数据库用户准备编译环境

su - postgres
sudo yum -y install \\
cmake \\
gcc \\
gcc-c++ \\
automake \\
autoconf \\
libtool \\
make \\
bison \\
flex \\
readline \\
readline-devel \\
ncurses-devel \\
zlib \\
zlib-devel \\
glib2-devel \\

（5）上传其余依赖组件—建议外部环境上编译后直接拷贝库到容器内

将压缩文件拷贝进Docker容器内：sudo docker XX/prom.zip 容器id或者容器名字:容器内路径

动态库放到/lib64目录下，然后执行ldconfig。

头文件编译的时候可使用–with-includes指定访问路径。

1：libmicrohttpd库安装

wget https://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-0.9.63.tar.gz -O libmicrohttpd-0.9.63.tar.gz
tar -xzf libmicrohttpd-0.9.63.tar.gz
cd libmicrohttpd-0.9.63
./configure --enable-shared=no --disable-doc --disable-examples --enable-https=no --disable-messages
sudo make
sudo make install

2：Prom工具：

******【源码编译】******

ubuntu: doc/he3db/prom.md · He3DB/He3PG - Gitee.com

Centos: 只能从官网下载源码编译

******【直接拷贝动态库】******

3：Rust：

Cargo编译环境：Linux上安装rust - 知乎 (zhihu.com)

******【源码编译】******

源码文件在https://gitee.com/he3db/he3fs/tree/devkv/pgfsgrpc 中，为dev_kv分支，后续可能会有变化，需要注意，可通过cargo build进行编译构建

******【直接拷贝动态库】******

He3DB源码编译

#创建源码目录

mkdir code

#创建编译后的二进制文件目录

mkdir app
cd code
git init
git remote add upstream [https://gitee.com/he3db/he3pg.git](https://gitee.com/he3db/he3pg.git)
git fetch upstream
git reset --hard upstream/dev\_performance

#编译debug版本

./configure --prefix=/home/postgres/app --with-includes=/home/postgres/tools/libprom/include:/home/postgres/tools/libpromhttp/include:/usr/lib64/glib-2.0/include --enable-debug --enable-depend --enable-cassert CFLAGS="-O0"
make && make install

#编译release版本

./configure --prefix=/home/postgres/app --with-includes=/home/postgres/tools/libprom/include:/home/postgres/tools/libpromhttp/include:/usr/lib64/glib-2.0/include
make && make install

Q&A

（1）编译报错找不到某个头文件

解决方法：

查找文件的目录： find / -name ‘文件名’ ，并将这个文件目录加入编译命令中

./configure --prefix=/home/postgres/app --with-includes=/home/postgres/tools/libprom/include:/home/postgres/tools/libpromhttp/include:/usr/lib64/glib-2.0/include --enable-debug --enable-depend --enable-cassert CFLAGS="-O0"

如果find查找没有这个依赖，需要安装的安装，需要拷贝的拷贝

（2）部署tikv

TiDB 数据库快速上手指南 | PingCAP 文档中心

添加环境变量: export PDADDR=“127.0.0.1:2397”

He3DB一主一备部署----将编译容器内生成的二进制文件拷贝到本地进行部署

脚本可参考：

#!/bin/bash
deploy_user=$(whoami)
if [ X"$deploy_user" = X"root" ]; then
	echo "error: can not deploy he3pg with root"
	exit 1
fi

#start tikv
env_file=~/.bash_profile
tikv_ip=127.0.0.1
tikv_port=1623

master_ip=127.0.0.1
master_port=1668
master_push_port=1669

standby_ip=127.0.0.1
standby_port=1670

path=/home/postgres/cj
data_path=${path}/data
app_path=${path}/app

cd ~
./.tiup/bin/tiup clean --all
nohup ./.tiup/bin/tiup playground v7.0.0 --mode tikv-slim --kv 1 --pd 1 --without-monitor --host ${tikv_ip} --pd.port ${tikv_port} --kv.config ./tikv.toml > ~/nohup_tikv.out 2>&1 &
sleep 5

#sed -i "/.*export\\s*PDADDR=/d" ${env_file}
#echo "export PDADDR=${tikv_ip}:${tikv_port}" >> ${env_file}
#source ${env_file}

export PDADDR=${tikv_ip}:${tikv_port}
echo "----------------print PDADDR-----------------------"
echo $PDADDR

#clean master and push
mkdir -p ${data_path}/pgdata_m1
rm -rf ${data_path}/pgdata_m1/*
mkdir -p ${data_path}/pgdata_mp
rm -rf ${data_path}/pgdata_mp/*
mkdir -p ${data_path}/pgdata_s1
rm -rf ${data_path}/pgdata_s1/*
rm -rf ${data_path}/logfile_*

export LD_LIBRARY_PATH=${app_path}/lib:$LD_LIBRARY_PATH
export PATH=${app_path}/bin:$PATH

#init push
cd ${app_path}

./bin/initdb -D ${data_path}/pgdata_mp

#copy to others
cp -a ${data_path}/pgdata_mp/* ${data_path}/pgdata_m1/
cp -a ${data_path}/pgdata_mp/* ${data_path}/pgdata_s1/

cd ${data_path}/pgdata_m1/
rm -rf base global postmaster.pid
ln -sf ${data_path}/pgdata_mp/base base
ln -sf ${data_path}/pgdata_mp/global global

#master
echo "host    replication     all             0.0.0.0/0               trust" >> ${data_path}/pgdata_m1/pg_hba.conf
echo "host    all             all             0.0.0.0/0               trust" >> ${data_path}/pgdata_m1/pg_hba.conf

echo "wal_recycle = off" >> ${data_path}/pgdata_m1/postgresql.conf
echo "fsync = off" >> ${data_path}/pgdata_m1/postgresql.conf
echo "checkpoint_timeout=30s" >> ${data_path}/pgdata_m1/postgresql.conf
#echo "wal_keep_size=10000" >> ${data_path}/pgdata_m1/postgresql.conf
echo "full_page_writes = off" >> ${data_path}/pgdata_m1/postgresql.conf
echo "hot_standby=off" >> ${data_path}/pgdata_m1/postgresql.conf
echo "push_standby=off" >> ${data_path}/pgdata_m1/postgresql.conf
echo "port="${master_port} >> ${data_path}/pgdata_m1/postgresql.conf
echo "lmdb_page_directory = '"${data_path}"/pgdata_m1/lmdb/page'" >> ${data_path}/pgdata_m1/postgresql.conf
echo "lmdb_wal_directory = '"${data_path}"/pgdata_m1/lmdb/wal'" >> ${data_path}/pgdata_m1/postgresql.conf
echo "listen_addresses = '*'" >> ${data_path}/pgdata_m1/postgresql.conf
echo "wal_consistency_checking='all'" >> ${data_path}/pgdata_m1/postgresql.conf

#push
echo "host    replication     all             0.0.0.0/0               trust" >> ${data_path}/pgdata_mp/pg_hba.conf
echo "host    all             all             0.0.0.0/0               trust" >> ${data_path}/pgdata_mp/pg_hba.conf

echo "primary_conninfo = 'application_name=push  dbname=postgres  user=repl password=He3@2023 host="${master_ip}" port="${master_port}" sslmode=disable sslcompression=0 gssencmode=disable target_session_attrs=any'" >> ${data_path}/pgdata_mp/postgresql.conf
echo "hot_standby = on" >> ${data_path}/pgdata_mp/postgresql.conf
echo "push_standby = on" >> ${data_path}/pgdata_mp/postgresql.conf
echo "wal_recycle=off" >> ${data_path}/pgdata_mp/postgresql.conf
echo "fsync = off" >> ${data_path}/pgdata_mp/postgresql.conf
#echo "wal_keep_size=10000" >> ${data_path}/pgdata_mp/postgresql.conf
echo "full_page_writes = off" >> ${data_path}/pgdata_mp/postgresql.conf
echo "port="${master_push_port} >> ${data_path}/pgdata_mp/postgresql.conf
echo "lmdb_page_directory = '"${data_path}"/pgdata_mp/lmdb/page'" >> ${data_path}/pgdata_mp/postgresql.conf
echo "lmdb_wal_directory = '"${data_path}"/pgdata_mp/lmdb/wal'" >> ${data_path}/pgdata_mp/postgresql.conf
echo "listen_addresses = '*'" >> ${data_path}/pgdata_mp/postgresql.conf
touch ${data_path}/pgdata_mp/standby.signal

#standby
echo "host    replication     all             0.0.0.0/0               trust" >> ${data_path}/pgdata_s1/pg_hba.conf
echo "host    all             all             0.0.0.0/0               trust" >> ${data_path}/pgdata_s1/pg_hba.conf

echo "primary_conninfo = 'application_name=standalone dbname=postgres user=repl password=He3@2023 host="${master_ip}" port="${master_port}" sslmode=disable sslcompression=0 gssencmode=disable target_session_attrs=any'" >> ${data_path}/pgdata_s1/postgresql.conf
echo "hot_standby = on" >> ${data_path}/pgdata_s1/postgresql.conf
echo "push_standby = off" >> ${data_path}/pgdata_s1/postgresql.conf
echo "wal_recycle = off" >> ${data_path}/pgdata_s1/postgresql.conf
echo "fsync = off" >> ${data_path}/pgdata_s1/postgresql.conf
#echo "wal_keep_size=10000" >> ${data_path}/pgdata_s1/postgresql.conf
echo "full_page_writes = off" >> ${data_path}/pgdata_s1/postgresql.conf
echo "port="${standby_port} >> ${data_path}/pgdata_s1/postgresql.conf
echo "lmdb_page_directory = '"${data_path}"/pgdata_s1/lmdb/page'" >> ${data_path}/pgdata_s1/postgresql.conf
echo "lmdb_wal_directory = '"${data_path}"/pgdata_s1/lmdb/wal'" >> ${data_path}/pgdata_s1/postgresql.conf
echo "listen_addresses = '*'" >> ${data_path}/pgdata_s1/postgresql.conf
echo "he3db_standalone = on" >> ${data_path}/pgdata_s1/postgresql.conf
touch ${data_path}/pgdata_s1/standby.signal

chmod -R 700 ${data_path}/pgdata_m1/
chmod -R 700 ${data_path}/pgdata_mp/
chmod -R 700 ${data_path}/pgdata_s1/

echo "start master"
cd ${app_path}
./bin/pg_ctl -D  ${data_path}/pgdata_m1/ -l ${data_path}/logfile_m1 start
sleep 1
./bin/psql -h ${master_ip} -p ${master_port} -d postgres -c "create role repl login replication encrypted password 'He3@2023';"
./bin/psql -h ${master_ip} -p ${master_port} -d postgres -c "grant pg_read_all_stats to repl;"

echo "start master push"
./bin/pg_ctl -D ${data_path}/pgdata_mp/ -l ${data_path}/logfile_mp start

echo "start standby"
./bin/pg_ctl -D ${data_path}/pgdata_s1 -l ${data_path}/logfile_s1 start

#./bin/psql -h ${master_ip} -p ${master_port} -d postgres -c "create database test;"

关键步骤：

二进制目录为/home/postgres/app/

#创建数据目录

mkdir /home/postgres/data
mkdir -p /home/postgres/data/pgdata\_master
mkdir -p /home/postgres/data/pgdata\_push
mkdir -p /home/postgres/data/pgdata\_standby
export LD\_LIBRARY\_PATH=/home/postgres/app/lib:$LD\_LIBRARY\_PATH
export PATH=/home/postgres/app/bin:$PATH

#init数据库

/home/postgres/app/bin/initdb -D /home/postgres/data/pgdata\_push

#拷贝去其他节点

cp -a /home/postgres/data/pgdata\_push/\* /home/postgres/data/pgdata\_master/

cp -a /home/postgres/data/pgdata\_push/\* /home/postgres/data/pgdata\_standby/

cd /home/postgres/data/pgdata\_master/

rm -rf base global postmaster.pid

ln -sf /home/postgres/data/pgdata\_push/base base

ln -sf /home/postgres/data/pgdata\_push/global global

#修改各节点postgresql.conf及pg_hba.conf文件

#master节点

cd /home/postgres/data/pgdata\_master

echo "host replication all 0.0.0.0/0 trust" >> pg\_hba.conf

echo "host all all 0.0.0.0/0 trust" >> pg\_hba.conf

echo "wal\_recycle = off" >> postgresql.conf

echo "fsync = off" >> postgresql.conf

echo "checkpoint\_timeout=30s" >> postgresql.conf

#echo "wal\_keep\_size=10000" >> postgresql.conf

echo "full\_page\_writes = off" >> postgresql.conf

echo "hot\_standby=off" >> postgresql.conf

echo "push\_standby=off" >> postgresql.conf

echo "port="${master\_port} >> postgresql.conf

echo "lmdb\_page\_directory=/home/postgres/data/pgdata\_master/lmdb/page" >> postgresql.conf

echo "lmdb\_wal\_directory = /home/postgres/data/pgdata\_master/wal/page" >> postgresql.conf

echo "listen\_addresses = '\*'" >> postgresql.conf

#push节点

cd /home/postgres/data/pgdata\_push

echo "host replication all 0.0.0.0/0 trust" >> pg\_hba.conf

echo "host all all 0.0.0.0/0 trust" >> pg\_hba.conf

echo "primary\_conninfo = 'application\_name=push dbname=postgres user=repl password=repl的密码 host="${master\_ip}" port="${master\_port}" sslmode=disable sslcompression=0 gssencmode=disable target\_session\_attrs=any'" >> postgresql.conf

echo "hot\_standby = on" >> postgresql.conf

echo "push\_standby = on" >> postgresql.conf

echo "wal\_recycle=off" >> postgresql.conf

echo "fsync = off" >> postgresql.conf

#echo "wal\_keep\_size=10000" >> postgresql.conf

echo "full\_page\_writes = off" >> postgresql.conf

echo "port="${push\_port} >> postgresql.conf

echo "lmdb\_page\_directory = /home/postgres/data/pgdata\_push/lmdb/page" >> postgresql.conf

echo "lmdb\_wal\_directory = /home/postgres/data/pgdata\_push/lmdb/wal" >> postgresql.conf

echo "listen\_addresses = '\*'" >> postgresql.conf

touch standby.signal

#standby节点

cd /home/postgres/data/pgdata\_standby

echo "host replication all 0.0.0.0/0 trust" >> ${data\_path}/pgdata\_s1/pg\_hba.conf

echo "host all all 0.0.0.0/0 trust" >> ${data\_path}/pgdata\_s1/pg\_hba.conf

echo "primary\_conninfo = 'application\_name=standalone dbname=postgres user=repl password=repl的密码 host="${master\_ip}" port="${master\_port}" sslmode=disable sslcompression=0 gssencmode=disable target\_session\_attrs=any'" >> postgresql.conf

echo "hot\_standby = on" >> postgresql.conf

echo "push\_standby = off" >> postgresql.conf

echo "wal\_recycle = off" >> postgresql.conf

echo "fsync = off" >> postgresql.conf

#echo "wal\_keep\_size=10000" >> postgresql.conf

echo "full\_page\_writes = off" >> postgresql.conf

echo "port="${standby\_port} >> postgresql.conf

echo "lmdb\_page\_directory=/home/postgres/data/pgdata\_standby/lmdb/page" >> postgresql.conf

echo "lmdb\_wal\_directory=/home/postgres/data/pgdata\_standby/lmdb/wal" >> postgresql.conf

echo "listen\_addresses = '\*'" >> postgresql.conf

echo "he3db\_standalone = on" >> postgresql.conf

touch standby.signal

#修改目录权限

cd /home/postgres/data/

chmod -R 700 pgdata\_master/

chmod -R 700 pgdata\_push/

chmod -R 700 pgdata\_standby/

#启动节点

echo "start master"

./bin/pg\_ctl -D ${data\_path}/pgdata\_m1/ -l ${data\_path}/logfile\_master start

./bin/psql -h ${master\_ip} -p ${master\_port} -d postgres -c "create role repl login replication encrypted password 'repl的密码';"

./bin/psql -h ${master\_ip} -p ${master\_port} -d postgres -c "grant pg\_read\_all\_stats to repl;"

echo "start master push"

./bin/pg\_ctl -D ${data\_path}/pgdata\_mp/ -l ${data\_path}/logfile\_push start

echo "start standby"

./bin/pg\_ctl -D ${data\_path}/pgdata\_s1 -l ${data\_path}/logfile\_standby start

cargo build环境准备

Cargo环境安装指导：Linux上安装rust - 知乎 (zhihu.com)

源代码路径：He3FS: He3FS是中移（苏州）软件技术有限公司基于JuiceFS专为He3DB打造的共享存储服务。 - Gitee.com

编译中的依赖：

sudo yum install -y openssl openssl-devel pkg-config perl protobuf-compiler

报错：

#![feature]` may not be used on the stable release channel 问题解决及设置Rust Channel的三种方式_一个不安分的程序员的博客-CSDN博客

he3db 数据库移动云数据库

「喜欢这篇文章，您的关注和赞赏是给作者最好的鼓励」

关注作者