暂无图片
暂无图片
1
暂无图片
暂无图片
暂无图片

MogDB数据库MOT内存表和MogDB基于磁盘表TPCC性能对比测试(CentOS_x86_64)

原创 怕晒的太阳 2022-09-06
1463

一.数据库安装准备

1.MogDB的单机安装见下链接:

https://www.modb.pro/db/476147

2.安装成功后,登录数据库设置






3.设置数据库参数

remote_read_mode = non_authentication
replication_type = 1
sync_config_strategy = none_node
recovery_max_workers = 20
max_connections = 3000
allow_concurrent_tuple_update = true
audit_enabled = off
cstore_buffers = 16MB
enable_alarm = off
enable_codegen = false
enable_data_replicate = off
full_page_writes = off
max_files_per_process = 100000
max_prepared_transactions = 2048
shared_buffers = 188GB    #内存的20%
use_workload_manager = off
wal_buffers = 1GB
work_mem = 1MB
transaction_isolation = 'read committed'
default_transaction_isolation = 'read committed'
synchronous_commit = off
fsync = on
maintenance_work_mem = 2GB
vacuum_cost_limit = 10000
autovacuum = on
autovacuum_mode = vacuum
autovacuum_max_workers = 20
autovacuum_naptime = 5s
autovacuum_vacuum_cost_delay = 10
update_lockwait_timeout = 20min
enable_mergejoin = off
enable_nestloop = off
enable_hashjoin = off
enable_material = off
wal_log_hints = off
log_duration = off
checkpoint_timeout = 15min
autovacuum_vacuum_scale_factor = 0.1
autovacuum_analyze_scale_factor = 0.02
enable_save_datachanged_timestamp = false
enable_double_write = on
enable_incremental_checkpoint = on
enable_opfusion = on
advance_xlog_file_num = 100
track_activities = off
enable_instr_track_wait = off
enable_instr_rt_percentile = off
track_counts = on
track_sql_count = off
enable_instr_cpu_timer = off
plog_merge_age = 0
session_timeout = 0
enable_instance_metric_persistent = off
enable_logical_io_statistics = off
enable_page_lsn_check = off
enable_user_metric_persistent = off
enable_xlog_prune = off
enable_resource_track = off
instr_unique_sql_count=0
wal_level = archive
hot_standby = off
hot_standby_feedback = off
client_min_messages = ERROR
log_min_messages = FATAL
enable_asp = off
enable_bbox_dump = off
bgwriter_flush_after = 32
wal_keep_segments = 1025
enable_bitmapscan = off
enable_seqscan = off
enable_beta_opfusion=on
enable_thread_pool = on
checkpoint_segments=8000
enable_stmt_track=false
bgwriter_delay = 5s
incremental_checkpoint_timeout = 5min
xloginsert_locks = 16
wal_file_init_num = 20
pagewriter_sleep = 10ms

4.操作系统参数

fs.aio-max-nr=1048576
fs.file-max= 76724600
kernel.sem = 50100 128256000 50100 2560
kernel.shmall = ‭148242432‬ # page, 80% MEM
kernel.shmmax= 607201001472
kernel.shmmni = 4096
net.core.netdev_max_backlog = 10000
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 4194304
net.core.somaxconn = 4096
net.ipv4.tcp_fin_timeout = 5
vm.overcommit_memory = 0
vm.swappiness = 1
net.ipv4.ip_local_port_range = 40000 65535
fs.nr_open = 20480000
vm.dirty_background_bytes=409600000
vm.nr_hugepages=51200
vm.mmap_min_addr=65536

5.设置core

echo "/home/core/core-%e-%p-%t" >/proc/sys/kernel/core_pattern

mkdir /home/core -p

chmod 777 /home/core

6.参数生效

sysctl -p 操作系统操作生效


gs_ctl restart -D /home/mogdb/data -Z single_node  重启数据库

二.benchmarkSQL5.0工作准备

相关软件下载地址
benchmarksql-5.0下载地址: https://udomain.dl.sourceforge.net/project/benchmarksql/benchmarksql-5.0.zip
R语言下载地址: https://mirror.bjtu.edu.cn/cran/src/base/R-3/R-3.6.3.tar.gz

以下安装R环境,是为了生成benchmarksql-5.0的监控信息。

#安装依赖软件包
yum install gcc glibc-headers gcc-c++ gcc-gfortran readline-devel libXt-devel pcre-devel libcurl libcurl-devel -y
yum install ncurses ncurses-devel autoconf automake zlib zlib-devel bzip2 bzip2-devel xz-devel -y
yum install java-1.8.0-openjdk ant -y
#安装R语言(generateReport.sh脚本需要)
yum install pango-devel pango libpng-devel cairo cairo-devel
## 使R语言支持png图片,否则报告生成有问题
tar -zxf R-3.6.3.tar.gz

cd R-3.6.3

./configure && make && make install  

如果没有benchmarksql-5.0的benchmarksql-5.0/dist/BenchmarkSQL-5.0.jar 不存在,需配置ant、java环境进行编译。

编译完成后,现在jdbc的jar放到benchmarksql-5.0/lib/postgres下。在run下配置数据库连接文件。

三.benchmarkSQL5.0在普通磁盘表的性能测试

1.导入数据:执行./runDatabaseBuild.sh mogdb.pg导入数据,执行前修改runDatabaseBuild.sh的权限chmod 755 run*.sh



2.500并发先预热2分钟。执行命令:./runBenchmark.sh mogdb.pg 执行tpcc。预热完成后,修改配置文件runMins=30,执行测试30分钟。


以下是运行30分钟前的数据表大小


以下是运行30分钟以后表大小


以下tpcc运行结果:


四.benchmarkSQL5.0在MOT内存表的性能测试

1.在测试内存表tpcc之前如下操作

MogDB=# drop database tpcc ;
DROP DATABASE
MogDB=# create database tpcc;
CREATE DATABASE
MogDB=# alter database tpcc owner to tpcc;

2.修改tpcc的表结构

操作以下操作替换原有的benchmarksql-5.0适合mot的操作



注意上图报错的是参数不存在,替换conf文件注意一下。

导入数据报错如下是由于参数max_process_memory设置较小,根据机器的实际情况进行调整参数重启数据库。


运行30分钟性能测试前表的情况


运行30分钟以后表的占用情况


2.500并发先预热2分钟。执行命令:./runBenchmark.sh mogdb.pg 执行tpcc。预热完成后,修改配置文件runMins=30,执行测试30分钟。


五.测试结果

结果经过以上测试发现,mot内存表的tpmC是普通表的tpmC 5-6倍。延迟比普通表小10倍左右。CPU的使用率比普通表大4倍。

mot的tpcc结果



普通表的tpcc测试结果:





数据库参数参照:

https://blog.csdn.net/weixin_53596073/article/details/123643316?spm=1001.2014.3001.5502

benchmarksql-5.0参照:

https://zhuanlan.zhihu.com/p/366861898

最后修改时间:2022-11-21 09:11:34
「喜欢这篇文章,您的关注和赞赏是给作者最好的鼓励」
关注作者
1人已赞赏
【版权声明】本文为墨天轮用户原创内容,转载时必须标注文章的来源(墨天轮),文章链接,文章作者等基本信息,否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容,欢迎发送邮件至:contact@modb.pro进行举报,并提供相关证据,一经查实,墨天轮将立刻删除相关内容。

文章被以下合辑收录

评论