暂无图片
暂无图片
暂无图片
暂无图片
暂无图片

MogDB例行维护表

原创 MogDB 2024-08-07
84

为了保证数据库的有效运行,数据库必须在插入/删除操作后,基于客户场景,定期做VACUUM FULL和ANALYZE,更新统计信息,以便获得更优的性能。

相关概念

使用VACUUM、VACUUM FULL和ANALYZE命令定期对每个表进行维护,主要有以下原因:

  • VACUUM FULL可回收已更新或已删除的数据所占据的磁盘空间,同时将小数据文件合并。
  • VACUUM对每个表维护了一个可视化映射来跟踪包含对别的活动事务可见的数组的页。一个普通的索引扫描首先通过可视化映射来获取对应的数组,来检查是否对当前事务可见。若无法获取,再通过堆数组抓取的方式来检查。因此更新表的可视化映射,可加速唯一索引扫描。
  • VACUUM可避免执行的事务数超过数据库阈值时,事务ID重叠造成的原有数据丢失。
  • ANALYZE可收集与数据库中表内容相关的统计信息。统计结果存储在系统表PG_STATISTIC中。查询优化器会使用这些统计数据,生成最有效的执行计划。

操作步骤

  1. 使用VACUUM或VACUUM FULL命令,进行磁盘空间回收。

    • VACUUM:

      对表执行VACUUM操作

      postgres=# VACUUM customer;
      复制
      VACUUM
      复制

      可以与数据库操作命令并行运行。(执行期间,可正常使用的语句: SELECT、INSERT、UPDATE和DELETE。不可正常使用的语句: ALTER TABLE)。

      对表分区执行VACUUM操作

      postgres=# VACUUM customer_par PARTITION ( P1 );
      复制
      VACUUM
      复制
    • VACUUM FULL:

      postgres=# VACUUM FULL customer;
      复制
      VACUUM
      复制

      需要向正在执行的表增加排他锁,且需要停止其他所有数据库操作。

  2. 使用ANALYZE语句更新统计信息。

    postgres=# ANALYZE customer;
    复制
    ANALYZE
    复制

    使用ANALYZE VERBOSE语句更新统计信息,并输出表的相关信息。

    postgres=# ANALYZE VERBOSE customer;
    复制
    ANALYZE
    复制

    也可以同时执行VACUUM ANALYZE命令进行查询优化。

    postgres=# VACUUM ANALYZE customer;
    复制
    VACUUM
    复制

    img 说明: VACUUM和ANALYZE会导致I/O流量的大幅增加,这可能会影响其他活动会话的性能。因此,建议通过"vacuum_cost_delay"参数设置。

  3. 删除表。

    postgres=# DROP TABLE customer; postgres=# DROP TABLE customer_par; postgres=# DROP TABLE part;
    复制

    当结果显示为如下信息,则表示删除成功。

    DROP TABLE
    复制

维护建议

  • 定期对部分大表做VACUUM FULL,在性能下降后为全库做VACUUM FULL,目前暂定每月做一次VACUUM FULL。
  • 定期对系统表做VACUUM FULL,主要是PG_ATTRIBUTE。
  • 启用系统自动清理线程(AUTOVACUUM)自动执行VACUUM和ANALYZE,回收被标识为删除状态的记录空间,并更新表的统计数据。

MogDB空闲空间回收机制简介

VACUUM的作用

空闲空间回收

由于MogDB的MVCC机制,表中会同时存放一行记录的多个历史版本,对于经常更新的表来说,会造成表空间的膨胀。为了解决这个问题,引入了空闲空间回收机制,并提供三种不同力度的空闲空间回收方式,用于不同场景下的空闲空间回收。根据触发时机和清理方式的不同,空闲空间回收分为三类:

  • 轻量级:轻量级空间回收在访问数据页时顺带清理页面中的过期元组,不清理索引,磁盘上的物理空间不会回收
  • 中量级:中量级空间回收有两种触发场景,一是后台线程自动发起的autovacuum,另一种是用户手动执行的VACUUM操作,会对索引和数据页面进行清理,特殊场景下会回收磁盘上的物理空间
  • 重量级:重量级空间回收主要由VACUUM FULL操作触发,会对指定的表进行彻底重建,回收磁盘上的物理空间

更新统计信息

中量级的空间回收会更新系统的统计信息,使planner能够计算出更准确的执行计划。

更新vm

在MogDB中,通过visibility map来标记数据页面的可见性,被vm标记的数据页面中没有过期元组,下一次VACUUM时会跳过该页面。此外,在进行索引扫描时,会首先检查页面的vm标记,以判断是否可进行indexonlyscan。

VACUUM的分类

轻量级

当查询扫描到某个数据页面时,会顺带清理页面中的过期元组。由于是顺带清理页面内容,因此只会删除过期元组本身,在页面内释放过期元组占用的空间,行指针则不做处理,避免在索引侧造成空指针或空引用。一个特殊情况是HOT场景,HOT场景是指对于表上的所有的索引,更新前后的索引键值均没有发生变化,因此对于更新后的元组只需要在数据页面中插入一条新记录,而不需要插入一条新的索引记录,通过HOT链来维护数据元组的新老版本。在轻量级空间回收时,只保留第一个版本的行指针,并将其重定向到第一个需要保留的元组版本的行指针。

img

中量级

MogDB提供VACUUM语句来让用户主动执行对某个astore表及其表上的索引进行空间回收,在astore中,新老版本的数据混合存储,进行VACUUM时会会首先对数据页面进行顺序扫描,判断哪些元组需要被清理,对于需要被清理的元组,会首先清理其对应的索引,然后再清理数据页面中的元组,从而避免索侧的空指针/空引用问题。

除了用户手动执行VACUUM操作外,MogDB还提供了自动的空闲空间回收功能,通过autovacuum周期性对表中的过期元组进行清理。与手动的VACUUM不同的是,autovacuum清理表和表的toast表是分开处理的,而手动vacuum是连续处理(该差异在MogDB 3.0.8最新补丁版本提供参数handle_toast_in_autovac供用户选择)。此外,autovacuum会被DDL操作中断,而手动的VACUUM操作则会阻塞DDL操作。

img

重量级

轻量级和中量级的空间回收都无法彻底释放磁盘上的物理空间,只能将数据页内的过期元组回收,释放其占用的空间,但是这些空间只能被当前表复用,无法将磁盘空间归还给操作系统。因此,MogDB提供了重量级的空闲空间回收VACUUM FULL操作。VACUUM FULL操作会将一个表中所有未过期元组从原数据页面中读取出来,然后重新紧密地插入到新的数据文件中,对于有索引的表,会在新的数据文件上重建索引,以彻底回收表中过期元组占用的空间。重量级空间回收操作的主体流程中只允许执行查询操作,而在提交流程,只读查询也会被阻塞。

img

示例及说明

自动空闲空间回收autovacuum

-- 1. update后等待连续的两次autovacuum MogDB=# create table tb(id int, name text); CREATE TABLE MogDB=# insert into tb values(0, 'asdf'); INSERT 0 1 MogDB=# update tb set id = 1 ; UPDATE 1 MogDB=# select relname, n_tup_ins, n_tup_del, n_live_tup, n_dead_tup, last_autovacuum, autovacuum_count from pg_stat_user_tables where relname = 'tb'; relname | n_tup_ins | n_tup_del | n_live_tup | n_dead_tup | last_autovacuum | autovacuum_count ---------+-----------+-----------+------------+------------+-----------------+------------------ tb | 1 | 0 | 1 | 1 | | 0 (1 row) MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 17095 | 17096 | 16386 | 258 | (0,2) 2 | 17096 | 0 | 32770 | 10242 | (0,2) (2 rows) MogDB=# select relname, n_tup_ins, n_tup_del, n_live_tup, n_dead_tup, last_autovacuum, autovacuum_count from pg_stat_user_tables where relname = 'tb'; relname | n_tup_ins | n_tup_del | n_live_tup | n_dead_tup | last_autovacuum | autovacuum_count ---------+-----------+-----------+------------+------------+-------------------------------+------------------ tb | 1 | 0 | 1 | 1 | 2023-12-24 15:36:23.018133+08 | 1 (1 row) MogDB=# select relname, n_tup_ins, n_tup_del, n_live_tup, n_dead_tup, last_autovacuum, autovacuum_count from pg_stat_user_tables where relname = 'tb'; relname | n_tup_ins | n_tup_del | n_live_tup | n_dead_tup | last_autovacuum | autovacuum_count ---------+-----------+-----------+------------+------------+-------------------------------+------------------ tb | 1 | 0 | 1 | 0 | 2023-12-24 15:37:23.079814+08 | 2 (1 row) -- 2. update后等待一次autovacuum,然后insert一条记录,查看xid是否被推进 MogDB=# create table tb(id int, name text); CREATE TABLE MogDB=# insert into tb values(0, 'asdf'); INSERT 0 1 MogDB=# update tb set id = 1 ; UPDATE 1 MogDB=# select relname, n_tup_ins, n_tup_del, n_live_tup, n_dead_tup, last_autovacuum, autovacuum_count from pg_stat_user_tables where relname = 'tb'; relname | n_tup_ins | n_tup_del | n_live_tup | n_dead_tup | last_autovacuum | autovacuum_count ---------+-----------+-----------+------------+------------+-----------------+------------------ tb | 1 | 0 | 1 | 1 | | 0 (1 row) MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 17088 | 17089 | 16386 | 258 | (0,2) 2 | 17089 | 0 | 32770 | 10242 | (0,2) (2 rows) MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 17088 | 17089 | 16386 | 1282 | (0,2) 2 | 17089 | 0 | 32770 | 10498 | (0,2) (2 rows) MogDB=# select relname, n_tup_ins, n_tup_del, n_live_tup, n_dead_tup, last_autovacuum, autovacuum_count from pg_stat_user_tables where relname = 'tb'; relname | n_tup_ins | n_tup_del | n_live_tup | n_dead_tup | last_autovacuum | autovacuum_count ---------+-----------+-----------+------------+------------+-------------------------------+------------------ tb | 1 | 0 | 1 | 1 | 2023-12-24 15:33:22.873957+08 | 1 (1 row) MogDB=# insert into tb values(0, 'asdf'); INSERT 0 1 MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 17088 | 17089 | 16386 | 1282 | (0,2) 2 | 17089 | 0 | 32770 | 10498 | (0,2) 3 | 17091 | 0 | 2 | 2050 | (0,3) (3 rows) MogDB=# select relname, n_tup_ins, n_tup_del, n_live_tup, n_dead_tup, last_autovacuum, autovacuum_count from pg_stat_user_tables where relname = 'tb'; relname | n_tup_ins | n_tup_del | n_live_tup | n_dead_tup | last_autovacuum | autovacuum_count ---------+-----------+-----------+------------+------------+-------------------------------+------------------ tb | 2 | 0 | 2 | 0 | 2023-12-24 15:34:22.897349+08 | 2 (1 row) MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | | | | | 2 | 17089 | 0 | 32770 | 10498 | (0,2) 3 | 17091 | 0 | 2 | 2306 | (0,3) (3 rows)
复制
  • 从第一个示例可以看到,update后,第二次对表执行autovacuum,表中的过期元组将被清理掉

  • 在第二个示例中,第一次insert的xid是17088,update的xid是17089,等待一次autovacuum之后再次进行insert的xid是17091,也就是autovacuum将xid从17089推进到了17090

手动空闲空间回收VACUUM

-- 1. update后直接vacuum MogDB=# create table tb(id int, name text); CREATE TABLE MogDB=# insert into tb values(0, 'asdf'); INSERT 0 1 MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 16702 | 0 | 2 | 2050 | (0,1) (1 row) MogDB=# select relname, n_tup_ins, n_tup_del, n_live_tup, n_dead_tup, last_autovacuum, autovacuum_count from pg_stat_user_tables where relname = 'tb'; relname | n_tup_ins | n_tup_del | n_live_tup | n_dead_tup | last_autovacuum | autovacuum_count ---------+-----------+-----------+------------+------------+-----------------+------------------ tb | 1 | 0 | 1 | 0 | | 0 (1 row) MogDB=# update tb set id = 1; UPDATE 1 MogDB=# select relname, n_tup_ins, n_tup_del, n_live_tup, n_dead_tup, last_autovacuum, autovacuum_count from pg_stat_user_tables where relname = 'tb'; relname | n_tup_ins | n_tup_del | n_live_tup | n_dead_tup | last_autovacuum | autovacuum_count ---------+-----------+-----------+------------+------------+-----------------+------------------ tb | 1 | 0 | 1 | 1 | | 0 (1 row) MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 16702 | 16703 | 16386 | 258 | (0,2) 2 | 16703 | 0 | 32770 | 10242 | (0,2) (2 rows) MogDB=# vacuum tb; VACUUM MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 16702 | 16703 | 16386 | 1282 | (0,2) 2 | 16703 | 0 | 32770 | 10498 | (0,2) (2 rows) MogDB=# select relname, n_tup_ins, n_tup_del, n_live_tup, n_dead_tup, last_autovacuum, autovacuum_count from pg_stat_user_tables where relname = 'tb'; relname | n_tup_ins | n_tup_del | n_live_tup | n_dead_tup | last_autovacuum | autovacuum_count ---------+-----------+-----------+------------+------------+-----------------+------------------ tb | 1 | 0 | 1 | 1 | | 0 (1 row) MogDB=# insert into tb values(0, 'asdf'); INSERT 0 1 MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 16702 | 16703 | 16386 | 1282 | (0,2) 2 | 16703 | 0 | 32770 | 10498 | (0,2) 3 | 16704 | 0 | 2 | 2050 | (0,3) (3 rows) MogDB=# vacuum tb; VACUUM MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | | | | | 2 | 16703 | 0 | 32770 | 10498 | (0,2) 3 | 16704 | 0 | 2 | 2306 | (0,3) (3 rows) MogDB=# select relname, n_tup_ins, n_tup_del, n_live_tup, n_dead_tup, last_autovacuum, autovacuum_count from pg_stat_user_tables where relname = 'tb'; relname | n_tup_ins | n_tup_del | n_live_tup | n_dead_tup | last_autovacuum | autovacuum_count ---------+-----------+-----------+------------+------------+-----------------+------------------ tb | 2 | 0 | 2 | 0 | | 0 (1 row) -- 2. update后执行其他操作推进一下xid,然后再vacuum MogDB=# create table tb(id int, name text); CREATE TABLE MogDB=# insert into tb values(0, 'asdf'); INSERT 0 1 MogDB=# update tb set id = 1; UPDATE 1 MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 16719 | 16720 | 16386 | 258 | (0,2) 2 | 16720 | 0 | 32770 | 10242 | (0,2) (2 rows) MogDB=# create table tbb(id int); CREATE TABLE MogDB=# vacuum tb; VACUUM MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | | | | | 2 | 16720 | 0 | 32770 | 10498 | (0,2) (2 rows) MogDB=# insert into tb values (0, 'asdf'); INSERT 0 1 MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | | | | | 2 | 16720 | 0 | 32770 | 10498 | (0,2) 3 | 16722 | 0 | 2 | 2050 | (0,3) (3 rows)
复制
  • 示例1中,update的xid是16702,update的xid是16703,update之后进行vacuum,然后进行了第二次insert,第二次的insert的xid是16704,也就是vacuum操作没有推进xid

表重建VACUUM FULL

-- 1. update后直接vacuum full MogDB=# create table tb(id int, name text); CREATE TABLE MogDB=# insert into tb values(0, 'asdf'); INSERT 0 1 MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 16707 | 0 | 2 | 2050 | (0,1) (1 row) MogDB=# update tb set id = 1; UPDATE 1 MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 16707 | 16708 | 16386 | 258 | (0,2) 2 | 16708 | 0 | 32770 | 10242 | (0,2) (2 rows) MogDB=# select relname, n_tup_ins, n_tup_del, n_live_tup, n_dead_tup, last_autovacuum, autovacuum_count from pg_stat_user_tables where relname = 'tb'; relname | n_tup_ins | n_tup_del | n_live_tup | n_dead_tup | last_autovacuum | autovacuum_count ---------+-----------+-----------+------------+------------+-----------------+------------------ tb | 1 | 0 | 1 | 1 | | 0 (1 row) MogDB=# vacuum full tb; VACUUM MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 16708 | 0 | 2 | 10498 | (0,1) 2 | 2 | 16708 | 2 | 1282 | (0,1) (2 rows) MogDB=# select relname, n_tup_ins, n_tup_del, n_live_tup, n_dead_tup, last_autovacuum, autovacuum_count from pg_stat_user_tables where relname = 'tb'; relname | n_tup_ins | n_tup_del | n_live_tup | n_dead_tup | last_autovacuum | autovacuum_count ---------+-----------+-----------+------------+------------+-----------------+------------------ tb | 1 | 0 | 1 | 1 | | 0 (1 row) MogDB=# vacuum full tb; VACUUM MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 2 | 0 | 2 | 10498 | (0,1) (1 row) MogDB=# select relname, n_tup_ins, n_tup_del, n_live_tup, n_dead_tup, last_autovacuum, autovacuum_count from pg_stat_user_tables where relname = 'tb'; relname | n_tup_ins | n_tup_del | n_live_tup | n_dead_tup | last_autovacuum | autovacuum_count ---------+-----------+-----------+------------+------------+-----------------+------------------ tb | 1 | 0 | 1 | 0 | | 0 (1 row) -- 2. update后先执行一次insert,再vacuum full MogDB=# create table tb(id int, name text); CREATE TABLE MogDB=# insert into tb values(0, 'asdf'); INSERT 0 1 MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 16713 | 0 | 2 | 2050 | (0,1) (1 row) MogDB=# update tb set id = 1; UPDATE 1 MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 16713 | 16714 | 16386 | 258 | (0,2) 2 | 16714 | 0 | 32770 | 10242 | (0,2) (2 rows) MogDB=# insert into tb values(0, 'asdf'); INSERT 0 1 MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 16713 | 16714 | 16386 | 258 | (0,2) 2 | 16714 | 0 | 32770 | 10242 | (0,2) 3 | 16715 | 0 | 2 | 2050 | (0,3) (3 rows) MogDB=# vacuum full tb; VACUUM MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 2 | 0 | 2 | 10498 | (0,1) 2 | 16715 | 0 | 2 | 2306 | (0,2) (2 rows) -- 3. 测试vacuum full是否会推进xid MogDB=# insert into tb values(0, 'asef'); INSERT 0 1 MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | | | | | 2 | 17096 | 0 | 32770 | 10498 | (0,2) 3 | 17100 | 0 | 2 | 2050 | (0,3) (3 rows) MogDB=# vacuum full tb; VACUUM MogDB=# insert into tb values(0, 'asef'); INSERT 0 1 MogDB=# select lp, t_xmin, t_xmax, t_infomask2, t_infomask, t_ctid from heap_page_items(get_raw_page('tb', 0)); lp | t_xmin | t_xmax | t_infomask2 | t_infomask | t_ctid ----+--------+--------+-------------+------------+-------- 1 | 2 | 0 | 2 | 10498 | (0,1) 2 | 17100 | 0 | 2 | 2306 | (0,2) 3 | 17102 | 0 | 2 | 2050 | (0,3) (3 rows)
复制
  • 示例1中,第一次vacuum full将过期元组的xmax设置为无效的xmax,将更新后的xmin设置为2,xmax设置为更新的xid,并推进xid,第二次vacuum full的时候将过期的元组清理掉,将更新后的元组xmin设置为2
  • 示例2中,update后通过insert将xid推进,然后进行vacuum full,所有过期元组均被清理掉
  • 示例3连续的两次insert之间进行了一次vacuum full,可以看到vacuum full将xid进行了推进

相关日志

autovacuum日志

-- 第一次autovacuum 2023-12-24 10:57:45.633 sxb postgres localhost 140347577464576 0[0:0#0] 0 [VACUUM] LOG: normally vacuum rel "public.tb" freeze 2000000000 OldestXmin 15540, FreezeLimit 3, freezeTableLimit 3 2023-12-24 10:57:47.952 sxb postgres localhost 140347577464576 0[0:0#0] 0 [VACUUM] LOG: "tb": found 0 removable, 100000 nonremovable row versions in 1031 out of 1031 pages 2023-12-24 10:57:47.952 sxb postgres localhost 140347577464576 0[0:0#0] 0 [VACUUM] DETAIL: 50000 dead row versions cannot be removed yet. There were 0 unused item pointers. 0 pages are entirely empty. CPU 0.00s/0.04u sec elapsed 2.31 sec. -- 第二次autovacuum 2023-12-24 10:58:15.634 sxb postgres localhost 140347577464576 0[0:0#0] 0 [VACUUM] LOG: normally vacuum rel "public.tb" freeze 2000000000 OldestXmin 15541, FreezeLimit 3, freezeTableLimit 3 2023-12-24 10:58:15.756 sxb postgres localhost 140347521890048 0[0:0#0] 0 [BACKEND] LOG: clean statement thread start 2023-12-24 10:58:17.950 sxb postgres localhost 140347577464576 0[0:0#0] 0 [VACUUM] LOG: "tb": found 50000 removable, 50000 nonremovable row versions in 1031 out of 1031 pages 2023-12-24 10:58:17.950 sxb postgres localhost 140347577464576 0[0:0#0] 0 [VACUUM] DETAIL: 0 dead row versions cannot be removed yet. There were 0 unused item pointers. 0 pages are entirely empty. CPU 0.00s/0.04u sec elapsed 2.31 sec.
复制

手动vacuum日志

-- 第一次vacuum 2023-12-24 10:50:12.308 sxb postgres [local] 139638516152064 0[0:0#0] 1125899906842767 [VACUUM] LOG: normally vacuum rel "public.tb" freeze -1 OldestXmin 15178, FreezeLimit 3, freezeTableLimit 3 2023-12-24 10:50:12.349 sxb postgres [local] 139638516152064 0[0:0#0] 1125899906842767 [VACUUM] LOG: "tb": found 0 removable, 100000 nonremovable row versions in 741 out of 741 pages 2023-12-24 10:50:12.349 sxb postgres [local] 139638516152064 0[0:0#0] 1125899906842767 [VACUUM] DETAIL: 50000 dead row versions cannot be removed yet. There were 0 unused item pointers. 0 pages are entirely empty. CPU 0.00s/0.04u sec elapsed 0.04 sec. -- 第二次vacuum 2023-12-24 10:52:12.909 sxb postgres [local] 139638516152064 0[0:0#0] 1125899906842783 [VACUUM] LOG: normally vacuum rel "public.tb" freeze -1 OldestXmin 15178, FreezeLimit 3, freezeTableLimit 3 2023-12-24 10:52:12.932 sxb postgres [local] 139638516152064 0[0:0#0] 1125899906842783 [VACUUM] LOG: "tb": found 0 removable, 100000 nonremovable row versions in 741 out of 741 pages 2023-12-24 10:52:12.932 sxb postgres [local] 139638516152064 0[0:0#0] 1125899906842783 [VACUUM] DETAIL: 50000 dead row versions cannot be removed yet. There were 0 unused item pointers. 0 pages are entirely empty. CPU 0.00s/0.02u sec elapsed 0.02 sec.
复制

vacuum full日志

-- 第一次vacuum full 2023-12-24 10:35:36.294 sxb postgres [local] 139638516152064 0[0:0#0] 1125899906842659 [BACKEND] LOG: vacuum test, OldestXmin: 15172 2023-12-24 10:35:36.294 sxb postgres [local] 139638516152064 0[0:0#0] 1125899906842659 [BACKEND] LOG: Relation tb(24642) [0/16245/24642] Swap files with Relation 24648 [0/16245/24648] xid 15173 -- 第二次vacuum full 2023-12-24 10:35:42.499 sxb postgres [local] 139638516152064 0[0:0#0] 1125899906842661 [BACKEND] LOG: vacuum test, OldestXmin: 15173 2023-12-24 10:35:42.499 sxb postgres [local] 139638516152064 0[0:0#0] 1125899906842661 [BACKEND] LOG: Relation tb(24642) [0/16245/24648] Swap files with Relation 24654 [0/16245/24654] xid 15174
复制

相关结论

  • autovacuum和手动vacuum的区别

    • 手动vacuum清理表和表的toast表是连续处理的,而autovacuum处理表和表的toast表之间有间隔。MogDB 3.0.8新增handle_toast_in_autovac参数,用于控制是否在autovacuum中清理表的同时清理该表对应的toast表
    • DDL操作会中断autovacuum对表的清理,而手动vacuum则会阻塞对正在进行清理的表进行的DDL操作
    • autovacuum执行后会推进OldestXmin,但是手动执行的vacuum不会推进OldestXmin
  • 什么情况下会清理页面中的过期元组,并更新pg_stat_user_tables中的n_dead_tup?

    • 页面中的过期元组足够老时,autovacuum、vacuum和vacuum full操作都会将这些过期元组清理掉
    • 由于autovacuum和vacuum full会推进xid,对于刚update/delete的老版本的元组,第一次autovacuum/vacuum full不会直接清理掉老版本的元组,但是会将老版本元组的informask相关位进行标记,第二次autovacuum/vacuum full时会将老版本元组清理掉
  • 如何理解足够老

    • MogDB中,根据可见性判断将元组分为HEAPTUPLE_DEAD、HEAPTUPLE_LIVE、HEAPTUPLE_RECENTLY_DEAD、HEAPTUPLE_INSERT_IN_PROGRESS、HEAPTUPLE_DELETE_IN_PROGRESS几种类型,其中,判断为HEAPTUPLE_RECENTLY_DEAD的元组,删除/更新老版本的事务已提交,但是在当前这个触发vacuum的事务中暂时还不能被删除,比如某一条元组,执行插入的xid=16000,xid=16001的事务将其更新到版本1,xid=16002的事务发起vacuum操作,此时,对老版本执行更新的事务16001虽然已经提交,但是对于当前这个vacuum来说,老版本属于HEAPTUPLE_RECENTLY_DEAD,当前这个vacuum操作不能直接对其进行清理。对于被判断为HEAPTUPLE_DEAD的元组,vacuum操作可直接将其进行清理

说明:本章节所有测试基于MogDB 5.0.5,日志中部分内容为修改源码后输出,正式版本中不包含。

「喜欢这篇文章,您的关注和赞赏是给作者最好的鼓励」
关注作者
【版权声明】本文为墨天轮用户原创内容,转载时必须标注文章的来源(墨天轮),文章链接,文章作者等基本信息,否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容,欢迎发送邮件至:contact@modb.pro进行举报,并提供相关证据,一经查实,墨天轮将立刻删除相关内容。

评论