dbops1.5 强力升级：支持 Prometheus 全家桶，监控 MySQL 更简单(三)

上一篇介绍基于部署了dbops绿色版ansible情况下，如何安装MySQL8.4一主一从架构。见

dbops1.5 强力升级：支持 Prometheus 全家桶，监控 MySQL 更简单(二)

接着第三篇，我要介绍如何给这两台一主一从的MySQL8.4安装部署 Node Exporter 和 MySQL Exporter，如何把 exporter 注册到 Prometheus，最后使 Grafana 出图。

1. 安装 Node Exporter 和 MySQL Exporter

1.1 配置

目前，MySQL Exporter 和 Node Exporter 的部署没有分开，使用同一个 playbook 安装。

cd usr/local/dbops/mysql_ansible/
cd playbooks
vi vars/var_exporter_install.yml

这里的大多数内容不建议修改，如果存在 MySQL 多实例混合部署的情况下，第二实例的 MySQL Exporter 的端口需要修改。

mysqld_exporter_port: 9104  # 第二实例改为 9105

其余建议使用默认值。

1.2 安装

ansible-playbook exporter_install.yml

1.3 确认信息

在执行 playbook 后，会输出以下信息，请确认无误后输入 confirm 继续。

TASK [Display the list of target hosts and additional information] *********************************************
ok: [192.168.199.131] => {
    "msg": [
        "Hosts to be affected by Deploy and configure node_exporter and mysqld_exporter: 192.168.199.131, 192.168.199.132",
        "DB type: mysql",
        "MySQL port: 3306",
        "MySQL version: 8.4.2",
        "Server specs: auto",
        "Roles to be executed: {'name': '../roles/node_exporter', 'condition': True}, {'name': '../roles/mysqld_exporter', 'condition': True}",
        "mysqld_exporter port: 9104",
        "mysqld_exporter install type: dbops",
        "mysqld_exporter install dir: database/mysql/exporter/3306",
        "node_exporter port: 9100",
        "node_exporter install type: dbops",
        "node_exporter install dir: usr/local/bin"
    ]
}

TASK [Prompt user for confirmation] ****************************************************************************
[Prompt user for confirmation]
This will perform Deploy and configure node_exporter and mysqld_exporter on the displayed hosts. Please type 'confirm' to continue or press Ctrl+C to cancel.:

输入 confirm 后，会开始安装。安装完成后，会输出以下信息：

RUNNING HANDLER [../roles/mysqld_exporter : Reload systemd] ****************************************************
ok: [192.168.199.132]
ok: [192.168.199.131]

RUNNING HANDLER [../roles/mysqld_exporter : Restart mysqld_exporter] *******************************************
changed: [192.168.199.132]
changed: [192.168.199.131]

PLAY RECAP *****************************************************************************************************
192.168.199.131            : ok=35   changed=10   unreachable=0    failed=0    skipped=24   rescued=0    ignored=0   
192.168.199.132            : ok=28   changed=9    unreachable=0    failed=0    skipped=19   rescued=0    ignored=0   

Playbook run took 0 days, 0 hours, 0 minutes, 22 seconds

至此,Node Exporter 和 MySQL Exporter 的安装就完成了。

2. 注册 Node Exporter 和 MySQL Exporter

在 usr/local/dbops/mysql_ansible/playbooks 目录下有一个 go 语言编写的 exporterregistrar 工具，它用于注册 Node Exporter 和 MySQL Exporter 到 Prometheus。

执行以下命令可以查看 exporterregistrar 工具的帮助信息和参考命令

cd /usr/local/dbops/mysql_ansible/playbooks
./exporterregistrar register --help

Register an exporter such as node_exporter or mysqld_exporter to Prometheus' targets configuration.

Examples:
  mysql server: 192.168.199.131
  prometheus server: 192.168.199.133
  prometheus server ssh user: root
  prometheus server ssh password: 123456

  # Register a MySQL exporter
  ./exporterregistrar register -t mysql -H 192.168.199.131 -s 192.168.199.133 -p 123456

  # Register a MySQL exporter with 3307 mysql port and 9105 metrics port
  ./exporterregistrar register -t mysql -H 192.168.199.131 --db-port 3307 -s 192.168.199.133 -P 9105 -p 123456  

  # Register a Node exporter
  ./exporterregistrar register -t node -H 192.168.199.131 -s 192.168.199.133 -p 123456

  # Register a MySQL exporter with custom labels
  ./exporterregistrar register -t mysql -H 192.168.199.131 -s 192.168.199.133 -p 123456 -f --region cn-sz
 
  # force update a MySQL exporter with custom labels
  ./exporterregistrar register -t mysql -H 192.168.199.131 -s 192.168.199.133 -p 123456 -f --region cn-bj --node-name node1

Usage:
  exporterregistrar register [flags]

Flags:
      --cluster string           Cluster label (default: mysql{$dbPort})
      --config-path string       Custom path for Prometheus configuration file.
                                 Default paths based on exporter type:
                                   - node_exporter: usr/local/prometheus/node_exporter_targets.yml
                                   - mysqld_exporter: usr/local/prometheus/mysqld_exporter_targets.yml
      --db-port string           Database port for labeling (default "3306")
  -f, --force                    Force update if target already exists (default false)
  -h, --help                     help for register
  -H, --host string              Hostname or IP address of the target exporter [required]
      --node-name string         Node name for labeling (default: same as host)
  -p, --password string          SSH password for Prometheus server [required]
  -P, --port string              Port of the target exporter (default: 9100 for node, 9104 for mysql/mysqld)
      --region string            Region label (default "cn")
      --replication-set string   Replication set label (default: mysql{$dbPort})
  -s, --server string            IP address of the Prometheus server [required]
      --ssh-port string          SSH port for Prometheus server (default "22")
  -t, --type string              Exporter type (node or mysql/mysqld) [required]
  -u, --user string              SSH username for Prometheus server (default "root")

Global Flags:
  -v, --verbose   enable verbose output

注册 3306 端口的 MySQL 实例到 Prometheus

192.168.199.131 主库
192.168.199.132 从库
192.168.199.133 Prometheus 服务器

主机

注册192.168.199.131 的 Node Exporter

./exporterregistrar register -t node -H 192.168.199.131 -s 192.168.199.133 -p ${prometheus_server_root_password}

注册192.168.199.132 的 Node Exporter

./exporterregistrar register -t node -H 192.168.199.132 -s 192.168.199.133 -p ${prometheus_server_root_password}

MySQL

注册192.168.199.131 的 MySQL Exporter

./exporterregistrar register -t mysql -H 192.168.199.131 -s 192.168.199.133 -p ${prometheus_server_password}

注册192.168.199.132 的 MySQL Exporter

./exporterregistrar register -t mysql -H 192.168.199.132 -s 192.168.199.133 -p ${prometheus_server_password}

执行效果如下:

[root@192-168-199-131 playbooks]# ./exporterregistrar register -t node -H 192.168.199.131 -s 192.168.199.133 -pgta@2015
Failed to parse YAML or no target groups found, initializing new target group
Adding new target: 192.168.199.131:9100
Exporter registered successfully.
[root@192-168-199-131 playbooks]# ./exporterregistrar register -t node -H 192.168.199.132 -s 192.168.199.133 -pgta@2015
Adding new target: 192.168.199.132:9100
Exporter registered successfully.
[root@192-168-199-131 playbooks]# ./exporterregistrar register -t mysql -H 192.168.199.132 -s 192.168.199.133 -pgta@2015
Failed to parse YAML or no target groups found, initializing new target group
Adding new target: 192.168.199.132:9104
Exporter registered successfully.
[root@192-168-199-131 playbooks]# ./exporterregistrar register -t mysql -H 192.168.199.131 -s 192.168.199.133 -pgta@2015
Adding new target: 192.168.199.131:9104
Exporter registered successfully.

我们再打开 grafana 的 web 页面，可以看到已经成功注册，并出图了。