TiCDC

本课练习中，您会在已有的 TiDB 数据库集群中增加 TiCDC 节点，之后进行数据的同步工作，之后将添加的 TiCDC 节点进行缩容。

部署 TiCDC

为原有 TiDB 数据库集群部署 TiCDC

在这个练习中,我们将在已有 TiDB 数据库集群中增加 TiCDC 节点,为下面的数据同步练习做准备｡

• 查看当前已有 TiDB 数据库集群状态

[root@db1 ~]# tiup cluster display tidb-test
复制

• 编辑扩容配置文件,准备将 TiCDC 节点 192.168.0.97 加入到集群中去

[root@db1 ~]# cat scale-out.yaml
cdc_servers:
  - host: 192.168.0.97
    port: 8300
    deploy_dir: "/tidb-deploy/cdc-8300"
    log_dir: "/tidb-deploy/cdc-8300/log"
复制

加入 1 个 TiCDC 节点, IP 为 192.168.0.97, 端口为 8300, 软件部署在 /tidb-deploy/cdc-8300 中,日志部署在 /tidb-deploy/cdc-8300/log 中｡ 因为只有一个集群节点,所以这个 TiCDC 集群不具备高可用性｡

• 使用 tiup 为原有 TiDB 数据库集群扩容 TiCDC 节点

[root@db1 ~]# tiup cluster scale-out tidb-test scale-out.yaml -uroot -p
复制

• 扩容完毕后,检查 TiDB 数据库集群状态｡

[root@db1 ~]# tiup cluster display tidb-test
复制

我们看到 TiCDC 集群的 ID 为 192.168.0.97:8300, Status(状态)为 UP,表示 TiCDC 部署成功｡

使用 tiup ctl:v5.0.0 cdc 检查 TiCDC 的状态

[root@db1 ~]# tiup ctl:v5.0.0 cdc capture list --pd=http://192.168.0.95:2379
The component `ctl` version v5.0.0 is not installed; downloading from repository.
download https://tiup-mirrors.pingcap.com/ctl-v5.0.0-linux-amd64.tar.gz 191.28 MiB / 191.28 MiB 100.00% 9.24 MiB/s
Starting component `ctl`: /root/.tiup/components/ctl/v5.0.0/ctl /root/.tiup/components/ctl/v5.0.0/ctl cdc capture list --pd=http://192.168.0.95:2379
[
  {
    "id": "c15deefa-1af4-4240-8402-634397de08d1",
    "is-owner": true,
    "address": "192.168.0.97:8300"
  }
]
复制

注意:

1. 第一次执行会自动下载此工具。
2. 命令中 --pd=http://192.168.0.95:2379, 可以是任何一个 PD 节点｡
3. “is-owner”: true 代表当 TiCDC 节点为 owner 节点｡

创建 TiDB 数据库到 MySQL 数据库的同步任务,并检查任务状态

在这个练习中,您将为已经为源库(上游) TiDB 数据库到 MySQL 数据库(下游)创建同步任务,开启数据同步,并检查任务状态｡

为 MySQL 数据库( 端口号为 3306 ) 加入时区信息

[root@db1 ~]# mysql_tzinfo_to_sql /usr/share/zoneinfo |mysql -uroot -p'!QAZ2wsx' -h127.0.0.1 -P3306 mysql
mysql: [Warning] Using a password on the command line interface can be insecure.
Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
复制

MySQL 数据库( 端口号为 3306 )创建数据库 test, 并创建表 T1 , 注意不插入数据

[root@db1 ~]# mysql -uroot -p'!QAZ2wsx' -h127.0.0.1 -P3306

mysql> create database test;
Query OK, 1 row affected (0.01 sec)

mysql> use test
Database changed
mysql> create table T1(id int primary key, name varchar(20));
Query OK, 0 rows affected (0.01 sec)

mysql> select * from T1;
Empty set (0.00 sec)
复制

准备 TiDB 数据库,创建数据库 test,并创建表 T1,注意不插入数据

[root@db1 ~]# mysql -h 192.168.0.97 -P 4000 -uroot

mysql> create database test;
Query OK, 0 rows affected (0.09 sec)

mysql> use test
Database changed
mysql> create table T1(id int primary key, name varchar(20));
Query OK, 0 rows affected (0.11 sec)

mysql> select * from T1;
Empty set (0.02 sec)
复制

进入到刚刚部署的 TiCDC 节点 192.168.0.97:8300,开启数据同步任务

[root@db1 ~]# cd /tidb-deploy/cdc-8300/bin/
[root@db1 bin]# ls -l
total 115988
-rwxr-xr-x. 1 tidb tidb 118768530 Apr  7  2021 cdc

[root@db1 bin]# ./cdc cli changefeed create \
--pd=http://192.168.0.95:2379 \
--sink-uri="mysql://root:QAZwsx123+@192.168.0.97:3306/" \
--changefeed-id="replication-task-1" \
--sort-engine="unified"
复制

如果出现 -bash: !QAZ2wsx: event not found ，需要将特殊字符转义，但是还是会出现以下问题，所以不建议密码有特殊字符“！”

[root@db1 bin]# ./cdc cli changefeed create \
> --pd=http://192.168.0.95:2379 \
> --sink-uri="mysql://root:QAZwsx123+@192.168.0.97:3306/" \
> --changefeed-id="replication-task-1" \
> --sort-engine="unified"
Create changefeed successfully!
ID: replication-task-1
Info: {"sink-uri":"mysql://root:QAZwsx123+@192.168.0.97:3306/","opts":{"_changefeed_id":"cli-verify"},"create-time":"2022-02-24T11:06:54.112674054+08:00","start-ts":431403044452696065,"target-ts":0,"admin-job-type":0,"sort-engine":"unified","sort-dir":"","config":{"case-sensitive":true,"enable-old-value":true,"force-replicate":false,"check-gc-safe-point":true,"filter":{"rules":["*.*"],"ignore-txn-start-ts":null,"ddl-allow-list":null},"mounter":{"worker-num":16},"sink":{"dispatchers":null,"protocol":"default"},"cyclic-replication":{"enable":false,"replica-id":0,"filter-replica-ids":null,"id-buckets":0,"sync-ddl":false},"scheduler":{"type":"table-number","polling-time":-1}},"state":"normal","history":null,"error":null,"sync-point-enabled":false,"sync-point-interval":600000000000,"creator-version":"v5.0.0"}
复制

对于刚刚创建的同步任务进行查询

[root@db1 bin]# ./cdc cli changefeed list --pd=http://192.168.0.95:2379
[
  {
    "id": "replication-task-1",
    "summary": {
      "state": "normal",
      "tso": 431403141852299265,
      "checkpoint": "2022-02-24 11:13:05.606",
      "error": null
    }
  }
]
复制

注意:
“state”: “normal” : 表示任务状态正常｡
“tso”: 431403141852299265 : 表示同步任务的时间戳信息｡
“checkpoint”: “2022-02-24 11:13:05.606” : 表示同步任务的时间｡

详细查询复制任务信息

[root@db1 bin]# ./cdc cli changefeed query --pd=http://192.168.0.95:2379 --changefeed-id=replication-task-1
{
  "info": {
    "sink-uri": "mysql://root:QAZwsx123+@192.168.0.97:3306/",
    "opts": {
      "_changefeed_id": "cli-verify"
    },
    "create-time": "2022-02-24T11:06:54.112674054+08:00",
    "start-ts": 431403044452696065,
    "target-ts": 0,
    "admin-job-type": 0,
    "sort-engine": "unified",
    "sort-dir": "",
    "config": {
      "case-sensitive": true,
      "enable-old-value": true,
      "force-replicate": false,
      "check-gc-safe-point": true,
      "filter": {
        "rules": [
          "*.*"
        ],
        "ignore-txn-start-ts": null,
        "ddl-allow-list": null
      },
      "mounter": {
        "worker-num": 16
      },
      "sink": {
        "dispatchers": null,
        "protocol": "default"
      },
      "cyclic-replication": {
        "enable": false,
        "replica-id": 0,
        "filter-replica-ids": null,
        "id-buckets": 0,
        "sync-ddl": false
      },
      "scheduler": {
        "type": "table-number",
        "polling-time": -1
      }
    },
    "state": "normal",
    "history": null,
    "error": null,
    "sync-point-enabled": false,
    "sync-point-interval": 600000000000,
    "creator-version": "v5.0.0"
  },
  "status": {
    "resolved-ts": 431403168708952065,
    "checkpoint-ts": 431403168184664065,
    "admin-job-type": 0
  },
  "count": 0,
  "task-status": [
    {
      "capture-id": "c15deefa-1af4-4240-8402-634397de08d1",
      "status": {
        "tables": {
          "126": {
            "start-ts": 431403044452696065,
            "mark-table-id": 0
          },
          "130": {
            "start-ts": 431403044452696065,
            "mark-table-id": 0
          },
          "154": {
            "start-ts": 431403044452696065,
            "mark-table-id": 0
          }
        },
        "operation": null,
        "admin-job-type": 0
      }
    }
  ]
}
复制

对于同步任务进行验证

• 登录 TiDB 数据库,查询刚刚创建的 test 数据库下面的表 T1,并且插入一行数据

[root@db1 ~]# mysql -h 192.168.0.97 -P 4000 -uroot

mysql> use test
Database changed

mysql> select * from T1;
Empty set (0.02 sec)

mysql> insert into T1 values(1,'Tom');
Query OK, 1 row affected (0.01 sec)

mysql> select * from T1;
+----+------+
| id | name |
+----+------+
|  1 | Tom  |
+----+------+
1 row in set (0.00 sec)
复制

• 登录 MySQL 数据库,查询 test 数据库下面的表 T1,发现数据库已经同步过去

[root@db1 ~]# mysql -uroot -p'!QAZ2wsx' -h127.0.0.1 -P3306

mysql> use test
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from T1;
+----+------+
| id | name |
+----+------+
|  1 | Tom  |
+----+------+
1 row in set (0.00 sec)
复制

缩容当前 TiCDC 节点

停止同步任务并删除同步任务

• 停止同步任务

[root@db1 bin]# ./cdc cli changefeed pause --pd=http://192.168.0.95:2379 --changefeed-id=replication-task-1

[root@db1 bin]# ./cdc cli changefeed list --pd=http://192.168.0.95:2379
[
  {
    "id": "replication-task-1",
    "summary": {
      "state": "stopped",
      "tso": 431403251834814465,
      "checkpoint": "2022-02-24 11:20:05.156",
      "error": null
    }
  }
]
复制

• 删除同步任务

[root@db1 bin]# ./cdc cli changefeed remove --pd=http://192.168.0.95:2379 --changefeed-id=replication-task-1

[root@db1 bin]# ./cdc cli changefeed list --pd=http://192.168.0.95:2379
[]
复制

使用如下命令缩容 TiCDC 集群

[root@db1 bin]# tiup cluster scale-in tidb-test --node 192.168.0.97:8300
复制