GTID概念:
GTID的工作原理:
从服务器连接到主服务器之后,把自己执行过的GTID (Executed_Gtid_Set: 即已经执行的事务编码)<SQL线程> 、获取到的GTID (Retrieved_Gtid_Set: 即从库已经接收到主库的事务编号) <IO线程>发给主服务器,主服务器把从服务器缺少的GTID及对应的transactions发过去补全即可。当主服务器挂掉的时候,找出同步最成功的那台从服务器,直接把它提升为主即可。如果硬要指定某一台不是最新的从服务器提升为主, 先change到同步最成功的那台从服务器, 等把GTID全部补全了,就可以把它提升为主了。
简单概述如下:
GTID的生命周期:gtid的生命周期对于配置和维护基于gtid的复制至关重要。
GTID的优缺点:
MySQL GTID复制部署过程如下。
属性 | 主数据库 | 从数据库 |
节点 | Mysql-Master01 | Mysql-Slave01 |
系统 | CentOS Linux release 7.5.1804 (Minimal) | CentOS Linux release 7.5.1804 (Minimal) |
内核 | 3.10.0-862.el7.x86_64 | 3.10.0-862.el7.x86_64 |
SELinux | setenforce 0 | disabled | setenforce 0 | disabled |
Firewlld | systemctl stop/disable firewalld | systemctl stop/disable firewalld |
IP地址 | 172.16.70.37 | 172.16.70.181 |
# 时间同步 [root@Mysql-Master01 ~]# yum install -y ntp [root@Mysql-Master01 ~]# systemctl start ntpd && systemctl enable ntpd [root@Mysql-Master01 ~]# timedatectl set-timezone Asia/Shanghai # yum安装MySQL5.7(默认最新版本) [root@Mysql-Master01 ~]# wget https://repo.mysql.com/yum/mysql-5.7-community/el/7/x86_64/mysql57-community-release-el7-10.noarch.rpm [root@Mysql-Master01 ~]# yum localinstall -y mysql57-community-release-el7-10.noarch.rpm [root@Mysql-Master01 ~]# yum repolist enabled | grep "mysql.*-community.*" [root@Mysql-Master01 ~]# yum install -y mysql-community-server [root@Mysql-Master01 ~]# mysql -V mysql Ver 14.14 Distrib 5.7.35, for Linux (x86_64) using EditLine wrapper # 启动MySQL [root@Mysql-Master01 ~]# systemctl start mysqld && systemctl enable Too few arguments. [root@Mysql-Master01 ~]# systemctl start mysqld && systemctl enable mysqld [root@Mysql-Master01 ~]# netstat -nutpl | grep mysql tcp6 0 0 :::3306 :::* LISTEN 2256/mysqld [root@Mysql-Master01 ~]# ps -ef | grep mysql mysql 2256 1 0 12:09 ? 00:00:08 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid # MySQL安全初始化 [root@Mysql-Master01 ~]# grep 'temporary password' /var/log/mysqld.log 2021-08-19T04:08:59.720748Z 1 [Note] A temporary password is generated for root@localhost: .!aTlyih4r2y [root@Mysql-Master01 ~]# mysql_secure_installation Securing the MySQL server deployment. Enter password for user root: # 输入MySQL初始密码 .!aTlyih4r2y The existing password for the user account root has expired. Please set a new password. New password: # 输入符合复杂密码策略的新密码 Re-enter new password: # 再次输入 The 'validate_password' plugin is installed on the server. The subsequent steps will run with the existing configuration of the plugin. Using existing password for root. Estimated strength of the password: 100 Change the password for root ? ((Press y|Y for Yes, any other key for No) : n # 上面以已经修改了,无需再修改 ... skipping. By default, a MySQL installation has an anonymous user, allowing anyone to log into MySQL without having to have a user account created for them. This is intended only for testing, and to make the installation go a bit smoother. You should remove them before moving into a production environment. Remove anonymous users? (Press y|Y for Yes, any other key for No) : y # 是否删除匿名用户 Success. Normally, root should only be allowed to connect from 'localhost'. This ensures that someone cannot guess at the root password from the network. Disallow root login remotely? (Press y|Y for Yes, any other key for No) : y # 是否禁用root远程登录 Success. By default, MySQL comes with a database named 'test' that anyone can access. This is also intended only for testing, and should be removed before moving into a production environment. Remove test database and access to it? (Press y|Y for Yes, any other key for No) : y # 是否删除test库和对test库的访问权限 - Dropping test database... Success. - Removing privileges on test database... Success. Reloading the privilege tables will ensure that all changes made so far will take effect immediately. Reload privilege tables now? (Press y|Y for Yes, any other key for No) : y # 是否刷新授权表使修改生效 Success. All done!
[root@Mysql-Master01 ~]# hostname -I 172.16.70.37 # 设置master01的my.cnf(必须在[mysqld]配置区域) [root@Mysql-Master01 ~]# cp /etc/my.cnf /etc/my.cnf_bak [root@Mysql-Master01 ~]# vim /etc/my.cnf [mysqld] ...... # 新增以下内容 #GTID server_id = 37 # master01服务器唯一ID,一般IP最后一段,主从不能重复 gtid_mode = on # 开启gtid模式 enforce_gtid_consistency = on # 强制gtid一直性,用于保证启动gitd后事务的安全 #binlog log_bin = master-bin # 开启bin-log,并可指定文件文件目录和前缀 log-slave-updates = 1 # 在从服务器进入主服务器传入过来的修改日志所使用,在Mysql5.7之前主从架构上使用gtid模式的话,必须使用此选项,在Mysql5.7取消了,会增加系统负载 binlog_format = row # 默认为mixed混合模式,更改为row复制,为了数据一致性,推荐采用row模式 sync-master-info = 1 # 同步master_info,任何事物提交以后都必须要把事务提交以后的二进制日志事件的位置对应的文件名称,记录到master_info中,下次启动自动读取,保证数据无丢失 sync_binlog = 1 # 表示binlog进行FSYNC刷盘,同时dump线程会在sync阶段后进行binlog传输 #relay log skip_slave_start = 1 # 跳过slave复制线程 # 重启MySQL [root@Mysql-Master01 ~]# systemctl restart mysqld # 登录MySQL [root@Mysql-Master01 ~]# mysql -p Enter password: mysql> show master status; # 查看master状态 +-------------------+----------+--------------+------------------+-------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +-------------------+----------+--------------+------------------+-------------------+ | master-bin.000002 | 154 | | | | +-------------------+----------+--------------+------------------+-------------------+ 1 row in set (0.00 sec) mysql> show global variables like '%uuid%'; +---------------+--------------------------------------+ | Variable_name | Value | +---------------+--------------------------------------+ | server_uuid | 2e12d559-00a3-11ec-9494-000c29ceb2c0 | +---------------+--------------------------------------+ 1 row in set (0.01 sec) mysql> show global variables like '%gtid%'; # 查看gtid功能是否开启 +----------------------------------+-------+ | Variable_name | Value | +----------------------------------+-------+ | binlog_gtid_simple_recovery | ON | | enforce_gtid_consistency | ON | | gtid_executed | | | gtid_executed_compression_period | 1000 | | gtid_mode | ON | | gtid_owned | | | gtid_purged | | | session_track_gtids | OFF | +----------------------------------+-------+ 8 rows in set (0.00 sec) mysql> show global variables like 'server_id'; # 查看服务器唯一ID +---------------+-------+ | Variable_name | Value | +---------------+-------+ | server_id | 37 | +---------------+-------+ 1 row in set (0.00 sec) mysql> show global variables like '%log_bin%'; # 查看binlog日志是否开启 +---------------------------------+---------------------------------+ | Variable_name | Value | +---------------------------------+---------------------------------+ | log_bin | ON | | log_bin_basename | /var/lib/mysql/master-bin | | log_bin_index | /var/lib/mysql/master-bin.index | | log_bin_trust_function_creators | OFF | | log_bin_use_v1_row_events | OFF | +---------------------------------+---------------------------------+ 5 rows in set (0.01 sec) mysql> GRANT REPLICATION SLAVE ON *.* TO 'backup'@'172.16.%.%' IDENTIFIED BY 'Backup@01'; # 建立backup账户并授权slave Query OK, 0 rows affected, 1 warning (0.00 sec) 语句说明: (1) replication slave为mysql同步的必须权限,此处不要授权all权限 (2) *.* 表示所有库所有表,也可以指定具体的库和表进行复制。例如mydb.tb1中,mydb为库名,tb1为表名 (3) 'backup'@'172.16.%.%' backup为同步账号。172.16.%.%为授权主机网段,使用了%表示允许整个172.16.0.0网段可以用backup这个用户访问数据库 (4) identified by 'Backup@01'; Backup@01为密码,实际环境下设置复杂密码 mysql> flush privileges; # 刷新权限 Query OK, 0 rows affected (0.01 sec) mysql> select user,host from mysql.user where user='backup'; # 查看是否存在backup用户 +--------+------------+ | user | host | +--------+------------+ | backup | 172.16.%.% | +--------+------------+ 1 row in set (0.01 sec) mysql> show grants for backup@'172.16.%.%'; # 查看backup用户授权 +---------------------------------------------------------+ | Grants for backup@172.16.%.% | +---------------------------------------------------------+ | GRANT REPLICATION SLAVE ON *.* TO 'backup'@'172.16.%.%' | +---------------------------------------------------------+ 1 row in set (0.00 sec) mysql> show master status; # 再次查看master状态 +-------------------+----------+--------------+------------------+------------------------------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +-------------------+----------+--------------+------------------+------------------------------------------+ | master-bin.000002 | 600 | | | 2e12d559-00a3-11ec-9494-000c29ceb2c0:1-2 | +-------------------+----------+--------------+------------------+------------------------------------------+ 1 row in set (0.00 sec) # 创建测试库mydb01 mysql> CREATE DATABASE IF NOT EXISTS mydb01; Query OK, 1 row affected (0.01 sec) mysql> INSERT INTO mydb01.tb01 VALUES(1,"zhangsan"),(2,"lisi"); Query OK, 2 rows affected (0.04 sec) Records: 2 Duplicates: 0 Warnings: 0 mysql> SELECT * FROM mydb01.tb01; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | | 2 | lisi | +----+----------+ 2 rows in set (0.00 sec) mysql> flush table with read lock; # 对主数据库锁表只读,防止导出数据库的时候有数据写入。unlock tables命令解除锁定 Query OK, 0 rows affected (0.01 sec) mysql> show variables like '%timeout%'; +-----------------------------+----------+ | Variable_name | Value | +-----------------------------+----------+ | connect_timeout | 10 | | delayed_insert_timeout | 300 | | have_statement_timeout | YES | | innodb_flush_log_at_timeout | 1 | | innodb_lock_wait_timeout | 50 | | innodb_rollback_on_timeout | OFF | | interactive_timeout | 28800 | # 自动解锁时间受本参数影响 | lock_wait_timeout | 31536000 | | net_read_timeout | 30 | | net_write_timeout | 60 | | rpl_stop_slave_timeout | 31536000 | | slave_net_timeout | 60 | | wait_timeout | 28800 | # 自动解锁时间受本参数影响 +-----------------------------+----------+ 13 rows in set (0.01 sec) mysql> show master status; # 锁表后查看主库状态 +-------------------+----------+--------------+------------------+------------------------------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +-------------------+----------+--------------+------------------+------------------------------------------+ | master-bin.000002 | 1300 | | | 2e12d559-00a3-11ec-9494-000c29ceb2c0:1-5 | +-------------------+----------+--------------+------------------+------------------------------------------+ 1 row in set (0.00 sec) # 备份mydb01数据库,发送至slave01 [root@Mysql-Master01 ~]# mysqldump --single-transaction --master-data=2 --triggers --routines --databases mydb01 -uroot -p > /root/mydb01.sql Enter password: 注意: mysql5.6使用mysqldump备份时,指定备份的具体库,使用--database mysql5.7使用mysqldump备份时,指定备份的具体库,使用--databases [root@Mysql-Master01 ~]# rsync -avz /root/mydb01.sql root@172.16.70.181:/root/ The authenticity of host '172.16.70.181 (172.16.70.181)' can't be established. ECDSA key fingerprint is SHA256:c/5+RMbf79VeNEzwtdtk9cvRoWIDDRg890ew82Hfj+g. ECDSA key fingerprint is MD5:41:ce:da:7c:7d:ce:93:ed:6f:c3:1d:81:6d:02:18:3b. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '172.16.70.181' (ECDSA) to the list of known hosts. root@172.16.70.181's password: # slave01服务器root密码 #导出数据完毕后,解锁主库 [root@Mysql-Master01 ~]# mysql -p Enter password: mysql> unlock tables; Query OK, 0 rows affected (0.00 sec)
[root@Mysql-Slave01 ~]# cp /etc/my.cnf /etc/my.cnf_bak [root@Mysql-Slave01 ~]# vim /etc/my.cnf [mysqld] #GTID server_id = 181 # 差异项 gtid_mode = on enforce_gtid_consistency = on #binlog log_bin = slave-bin # 差异项 log-slave-updates = 1 binlog_format = row sync-master-info = 1 sync_binlog = 1 #relay log skip_slave_start = 1 read_only = on # 差异项;使从服务器只能进行读取操作,此参数对超级用户无效,并且不会影响从服务器的复制 # 重启MySQL [root@Mysql-Slave01 ~]# systemctl restart mysqld # 登录MySQL [root@Mysql-Slave01 ~]# mysql -p Enter password: mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | +--------------------+ 4 rows in set (0.01 sec) mysql> source /root/mydb01.sql; mysql> select * from mydb01.tb01; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | | 2 | lisi | +----+----------+ 2 rows in set (0.00 sec) mysql> show variables like 'server_id'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | server_id | 181 | +---------------+-------+ 1 row in set (0.01 sec) mysql> show variables like '%log_bin%'; +---------------------------------+---------------------------------+ | Variable_name | Value | +---------------------------------+---------------------------------+ | log_bin | ON | | log_bin_basename | /var/lib/mysql/master-bin | | log_bin_index | /var/lib/mysql/master-bin.index | | log_bin_trust_function_creators | OFF | | log_bin_use_v1_row_events | OFF | | sql_log_bin | ON | +---------------------------------+---------------------------------+ 6 rows in set (0.00 sec) # 配置主从同步指令 change master mysql> stop slave; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> change master to master_host='172.16.70.37',master_port=3306,master_user='backup',master_password='Backup@01',master_auto_position=1; Query OK, 0 rows affected, 2 warnings (0.03 sec) 参数说明:(提示:字符串用单引号括起来,数值不用引号,注意内容前后不能有空格。) change master to master_host='172.16.70.37' # master主库IP master_port=3306 # 数据库端口号 master_user='backup' # master上创建用于复制的用户 master_password='Backup@01' # 复制用户的密码 master_auto_position=1 # gtid复制必须设置此项 ------------------------------------------------------------------------- # 上述操作的原理实际上是把用户密码等信息写入从库新的master.info文件中 [root@Mysql-Slave01 ~]# cat /var/lib/mysql/*.info 25 4 172.16.70.37 backup Backup@01 3306 60 ... -------------------------------------------------------------------------- mysql> start slave; Query OK, 0 rows affected (0.01 sec) mysql> show slave status \G; *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.16.70.37 Master_User: backup Master_Port: 3306 Connect_Retry: 60 Master_Log_File: master-bin.000002 Read_Master_Log_Pos: 1300 Relay_Log_File: Mysql-Slave01-relay-bin.000002 Relay_Log_Pos: 417 Relay_Master_Log_File: master-bin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 1300 Relay_Log_Space: 632 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 37 Master_UUID: 2e12d559-00a3-11ec-9494-000c29ceb2c0 Master_Info_File: /var/lib/mysql/master.info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: Executed_Gtid_Set: 2e12d559-00a3-11ec-9494-000c29ceb2c0:1-5 # master主数据库的GTID Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec) ERROR: No query specifie 如上,当IO和SQL线程的状态均为Yes,则表示主从已实现同步了!(主从同步是否成功,最关键的为下面的3项状态参数) Slave_IO_Running: Yes,这个时I/O线程状态,I/O线程负责从从库到主库读取binlog日志,并写入从库的中继日志,状态为Yes表示I/O线程工作正常。 Slave_SQL_Running: Yes,这个是SQL线程状态,SQL线程负责读取中继日志(relay-log)中的数据并转换为SQL语句应用到从数据库中,状态为Yes表示I/O线程工作正常。 Seconds_Behind_Master:0,这个是复制过程中从库比主库延迟的秒数,这个参数极度重要,但企业里更准确地判断主从延迟的方法为:在主库写时间戳,然后从库读取时间戳,和当前数据库时间进行比较,从而认定是否延迟。 =============================================================================== # 再次回到master01机,查看master状态 mysql> show slave hosts; +-----------+------+------+-----------+--------------------------------------+ | Server_id | Host | Port | Master_id | Slave_UUID | +-----------+------+------+-----------+--------------------------------------+ | 181 | | 3306 | 37 | 343429c9-00a3-11ec-a6e0-000c29191ffb | +-----------+------+------+-----------+--------------------------------------+ 1 row in set (0.00 sec
# 在172.16.70.37(master01)的主数据库插入新数据 mysql> insert into mydb01.tb01 values(11,"chenqi"),(12,"huangba"); Query OK, 2 rows affected (0.00 sec) Records: 2 Duplicates: 0 Warnings: 0 mysql> select * from mydb01.tb01; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | | 2 | lisi | | 11 | chenqi | | 12 | huangba | +----+----------+ 4 rows in set (0.00 sec) # 然后到172.16.70.181(slave01)上查看是否自动同步 mysql> select * from mydb01.tb01; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | | 2 | lisi | | 11 | chenqi | | 12 | huangba | +----+----------+ 4 rows in set (0.00 sec) 至此,MySQL主从复制(GTID)已经实现!
MySqL的读写分离器MySql-Proxy
Mysql-Proxy读写分离部署。
属性 | 主数据库 | 从数据库 | 中间件 |
节点 | Mysql-Master01 | Mysql-Slave01 | Mysql-Proxy01 |
系统 | CentOS Linux release 7.5.1804 (Minimal) | CentOS Linux release 7.5.1804 (Minimal) | CentOS Linux release 7.5.1804 (Minimal) |
内核 | 3.10.0-862.el7.x86_64 | 3.10.0-862.el7.x86_64 | 3.10.0-862.el7.x86_64 |
SELinux | setenforce 0 | disabled | setenforce 0 | disabled | setenforce 0 | disabled |
Firewlld | systemctl stop/disable firewalld | systemctl stop/disable firewalld | systemctl stop/disable firewalld |
IP地址 | 172.16.70.37 | 172.16.70.181 | 172.16.70.182 |
# 配置监控后端MySQL节点。(在master主数据节点上创建一个用于监控的用户名(只需在master上创建即可,因为会复制到slave上),这个用户名只需具有USAGE权限即可) [root@Mysql-Master01 ~]# mysql -p Enter password: mysql> create user monitor@'172.16.70.%' identified by 'Monitor@01'; Query OK, 0 rows affected (0.01 sec) mysql> grant replication client on *.* to monitor@'172.16.70.%'; Query OK, 0 rows affected (0.01 sec) mysql> grant all on *.* to root@'172.16.70.%' identified by 'Password@01'; Query OK, 0 rows affected, 1 warning (0.00 sec) # 配置mysql_users mysql> grant all on *.* to stnduser@'172.16.70.%' identified by 'Password@01'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> show grants for monitor@'172.16.70.%'; +------------------------------------------------------------+ | Grants for monitor@172.16.70.% | +------------------------------------------------------------+ | GRANT REPLICATION CLIENT ON *.* TO 'monitor'@'172.16.70.%' | +------------------------------------------------------------+ 1 row in set (0.00 sec) mysql> flush privileges; Query OK, 0 rows affected (0.01 sec)
# CentOS 添加仓库(根据需要选择适合版本,这里选择proxysql-2.1.x 版本) [root@Mysql-Proxy01 ~]# vim /etc/yum.repos.d/proxysql.repo [proxysql_repo] name= ProxySQL YUM repository baseurl=http://repo.proxysql.com/ProxySQL/proxysql-2.1.x/centos/$releasever gpgcheck=1 gpgkey=http://repo.proxysql.com/ProxySQL/repo_pub_key # 执行安装 [root@Mysql-Proxy01 ~]# yum install proxysql -y [root@Mysql-Proxy01 ~]# proxysql -V ProxySQL version 2.1.1-40-g1c2b7e4, codename Truls # 启动proxsql [root@Mysql-Proxy01 ~]# systemctl start proxysql && systemctl enable proxysql # 监听两默认端口:6032,6033 [root@Mysql-Proxy01 ~]# systemctl start proxysql && systemctl enable proxysql [root@Mysql-Proxy01 ~]# netstat -nutpl | grep proxy tcp 0 0 0.0.0.0:6032 0.0.0.0:* LISTEN 2885/proxysql tcp 0 0 0.0.0.0:6033 0.0.0.0:* LISTEN 2885/proxysql [root@Mysql-Proxy01 ~]# ps -ef | grep proxy proxysql 2884 1 0 Aug23 ? 00:00:00 /usr/bin/proxysql --idle-threads -c /etc/proxysql.cnf proxysql 2885 2884 0 Aug23 ? 00:06:08 /usr/bin/proxysql --idle-threads -c /etc/proxysql.cnf # 默认配置文件/etc/proxysql.cnf [root@Mysql-Proxy01 ~]# grep -Ev '#|^$' /etc/proxysql.cnf datadir="/var/lib/proxysql" admin_variables= { admin_credentials="admin:admin" # 管理接口默认账号:密码(可修改) mysql_ifaces="0.0.0.0:6032" # 客户端接口端口 } mysql_variables= { threads=4 max_connections=2048 default_query_delay=0 default_query_timeout=36000000 have_compress=true poll_timeout=2000 interfaces="0.0.0.0:6033" default_schema="information_schema" stacksize=1048576 server_version="5.5.30" connect_timeout_server=3000 monitor_username="monitor" monitor_password="monitor" monitor_history=600000 monitor_connect_interval=60000 monitor_ping_interval=10000 monitor_read_only_interval=1500 monitor_read_only_timeout=500 ping_interval_server_msec=120000 ping_timeout_server=500 commands_stats=true sessions_sort=true connect_retries_on_failure=10 } mysql_servers = ( ) mysql_users: ( ) mysql_query_rules: ( ) scheduler= ( ) mysql_replication_hostgroups= ( ) # 登录管理界面(使用默认凭据) [root@Mysql-Proxy01 ~]# mysql -uadmin -padmin -P6032 -h127.0.0.1 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.5.30 (ProxySQL Admin Module) Copyright (c) 2000, 2021, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +-----+---------------+-------------------------------------+ | seq | name | file | +-----+---------------+-------------------------------------+ | 0 | main | | | 2 | disk | /var/lib/proxysql/proxysql.db | | 3 | stats | | | 4 | monitor | | | 5 | stats_history | /var/lib/proxysql/proxysql_stats.db | +-----+---------------+-------------------------------------+ 5 rows in set (0.00 sec) mysql> show tables from main; +----------------------------------------------------+ | tables | +----------------------------------------------------+ | global_variables | | mysql_aws_aurora_hostgroups | | mysql_collations | | mysql_firewall_whitelist_rules | | mysql_firewall_whitelist_sqli_fingerprints | | mysql_firewall_whitelist_users | | mysql_galera_hostgroups | | mysql_group_replication_hostgroups | | mysql_query_rules | | mysql_query_rules_fast_routing | | mysql_replication_hostgroups | | mysql_servers | | mysql_users | | proxysql_servers | | restapi_routes | | runtime_checksums_values | | runtime_global_variables | | runtime_mysql_aws_aurora_hostgroups | | runtime_mysql_firewall_whitelist_rules | | runtime_mysql_firewall_whitelist_sqli_fingerprints | | runtime_mysql_firewall_whitelist_users | | runtime_mysql_galera_hostgroups | | runtime_mysql_group_replication_hostgroups | | runtime_mysql_query_rules | | runtime_mysql_query_rules_fast_routing | | runtime_mysql_replication_hostgroups | | runtime_mysql_servers | | runtime_mysql_users | | runtime_proxysql_servers | | runtime_restapi_routes | | runtime_scheduler | | scheduler | +----------------------------------------------------+ 32 rows in set (0.00 sec) 说明重要字段含义: global_variables 设置变量,包括监听的端口、管理账号等。 mysql_collations 相关字符集和校验规则。 mysql_query_rules 定义查询路由规则。 mysql> show tables from monitor; +--------------------------------------+ | tables | +--------------------------------------+ | mysql_server_aws_aurora_check_status | | mysql_server_aws_aurora_failovers | | mysql_server_aws_aurora_log | | mysql_server_connect_log | | mysql_server_galera_log | | mysql_server_group_replication_log | | mysql_server_ping_log | | mysql_server_read_only_log | | mysql_server_replication_lag_log | +--------------------------------------+ 9 rows in set (0.00 sec) 注意:runtime_开头的是运行时的配置,这些是不能修改的。要修改ProxySQL的配置,需要修改了非runtime_表,修改后必须执行LOAD ... TO RUNTIME 才能加载到RUNTIME生效,执行save ... to disk才能将配置持久化保存到磁盘。 # 添加MySQL节点,使用insert语句添加主机到mysql_servers表中,其中:hostgroup_id 1 表示写组,2表示读组 mysql> insert into mysql_servers(hostgroup_id,hostname,port) values(1,'172.16.70.37',3306); Query OK, 1 row affected (0.00 sec) mysql> insert into mysql_servers(hostgroup_id,hostname,port) values(1,'172.16.70.181',3306); Query OK, 1 row affected (0.00 sec) mysql> select * from mysql_servers; # 查看insert结果 +--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+ | hostgroup_id | hostname | port | gtid_port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment | +--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+ | 1 | 172.16.70.37 | 3306 | 0 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | | | 1 | 172.16.70.181 | 3306 | 0 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | | +--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+ 2 rows in set (0.00 sec) # 监控用户的凭据添加到 ProxySQL mysql> update global_variables set variable_value='monitor' where variable_name='mysql-monitor_username'; Query OK, 0 rows affected (0.00 sec) mysql> update global_variables set variable_value='Monitor@01' where variable_name='mysql-monitor_password'; Query OK, 0 rows affected (0.01 sec) mysql> select * from global_variables where variable_name like 'mysql-monitor_%'; +--------------------------------------------------------------+----------------+ | variable_name | variable_value | +--------------------------------------------------------------+----------------+ | mysql-monitor_enabled | true | | mysql-monitor_connect_timeout | 600 | | mysql-monitor_ping_max_failures | 3 | | mysql-monitor_ping_timeout | 1000 | | mysql-monitor_read_only_max_timeout_count | 3 | | mysql-monitor_replication_lag_interval | 10000 | | mysql-monitor_replication_lag_timeout | 1000 | | mysql-monitor_replication_lag_count | 1 | | mysql-monitor_groupreplication_healthcheck_interval | 5000 | | mysql-monitor_groupreplication_healthcheck_timeout | 800 | | mysql-monitor_groupreplication_healthcheck_max_timeout_count | 3 | | mysql-monitor_groupreplication_max_transactions_behind_count | 3 | | mysql-monitor_galera_healthcheck_interval | 5000 | | mysql-monitor_galera_healthcheck_timeout | 800 | | mysql-monitor_galera_healthcheck_max_timeout_count | 3 | | mysql-monitor_replication_lag_use_percona_heartbeat | | | mysql-monitor_query_interval | 60000 | | mysql-monitor_query_timeout | 100 | | mysql-monitor_slave_lag_when_null | 60 | | mysql-monitor_threads_min | 8 | | mysql-monitor_threads_max | 128 | | mysql-monitor_threads_queue_maxsize | 128 | | mysql-monitor_wait_timeout | true | | mysql-monitor_writer_is_also_reader | true | | mysql-monitor_username | monitor | # 自定义项 | mysql-monitor_password | Monitor@01 | # 自定义项 | mysql-monitor_history | 600000 | | mysql-monitor_connect_interval | 60000 | | mysql-monitor_ping_interval | 10000 | | mysql-monitor_read_only_interval | 1500 | | mysql-monitor_read_only_timeout | 500 | +--------------------------------------------------------------+----------------+ 31 rows in set (0.00 sec) # 激活配置,并保存到磁盘 mysql> load mysql servers to runtime; Query OK, 0 rows affected (0.00 sec) mysql> save mysql servers to disk; Query OK, 0 rows affected (0.06 sec) # 指定一对 READER 和 WRITER 主机组进行配置(写组id:1 ; 读组id:2) mysql> insert into mysql_replication_hostgroups (writer_hostgroup,reader_hostgroup,comment) values (1,2,'cluster1'); Query OK, 1 row affected (0.00 sec) mysql> select * from mysql_replication_hostgroups; +------------------+------------------+------------+----------+ | writer_hostgroup | reader_hostgroup | check_type | comment | +------------------+------------------+------------+----------+ | 1 | 2 | read_only | cluster1 | +------------------+------------------+------------+----------+ 1 row in set (0.00 sec) mysql> load mysql servers to runtime; Query OK, 0 rows affected (0.00 sec) # 后台健康检查 mysql> show tables from monitor; +------------------------------------+ | tables | +------------------------------------+ | mysql_server_connect_log | | mysql_server_group_replication_log | | mysql_server_ping_log | | mysql_server_read_only_log | | mysql_server_replication_lag_log | +------------------------------------+ 5 rows in set (0.00 sec) mysql> select * from monitor.mysql_server_connect_log order by time_start_us desc limit 2; # 对心跳信息的监控(connect_log) +---------------+------+------------------+-------------------------+---------------+ | hostname | port | time_start_us | connect_success_time_us | connect_error | +---------------+------+------------------+-------------------------+---------------+ | 172.16.70.37 | 3306 | 1629795694640282 | 3725 | NULL | | 172.16.70.181 | 3306 | 1629795693592247 | 4307 | NULL | +---------------+------+------------------+-------------------------+---------------+ 2 rows in set (0.00 sec) mysql> select * from monitor.mysql_server_ping_log order by time_start_us desc limit 2; # 对心跳信息的监控(ping_log) +---------------+------+------------------+----------------------+------------+ | hostname | port | time_start_us | ping_success_time_us | ping_error | +---------------+------+------------------+----------------------+------------+ | 172.16.70.181 | 3306 | 1629795783687130 | 1114 | NULL | | 172.16.70.37 | 3306 | 1629795783546617 | 716 | NULL | +---------------+------+------------------+----------------------+------------+ 2 rows in set (0.00 sec mysql> select * from mysql_servers; +--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+ | hostgroup_id | hostname | port | gtid_port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment | +--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+ | 1 | 172.16.70.37 | 3306 | 0 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | | | 2 | 172.16.70.181 | 3306 | 0 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | | +--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+ 2 rows in set (0.00 sec) # 配置mysql_users mysql> insert into mysql_users(username,password,default_hostgroup) values('root','Password@01',1); Query OK, 1 row affected (0.01 sec) mysql> insert into mysql_users(username,password,default_hostgroup) values('stnduser','Password@01',1); Query OK, 1 row affected (0.00 sec) mysql_users表有不少字段,最主要的三个字段为username、password和default_hostgroup: username:前端连接ProxySQL,以及ProxySQL将SQL语句路由给MySQL所使用的用户名。 password:用户名对应的密码。可以是明文密码,也可以是hash密码。如果想使用hash密码,可以先在某个MySQL节点上执行select password(PASSWORD),然后将加密结果复制到该字段。 default_hostgroup:该用户名默认的路由目标。例如,指定root用户的该字段值为10时,则使用root用户发送的SQL语句默认情况下将路由到hostgroup_id=10组中的某个节点。 mysql> select * from mysql_users\G *************************** 1. row *************************** username: root password: Password@01 active: 1 use_ssl: 0 default_hostgroup: 1 default_schema: NULL schema_locked: 0 transaction_persistent: 1 fast_forward: 0 backend: 1 frontend: 1 max_connections: 10000 attributes: comment: *************************** 2. row *************************** username: stnduser password: Password@01 active: 1 use_ssl: 0 default_hostgroup: 1 default_schema: NULL schema_locked: 0 transaction_persistent: 1 fast_forward: 0 backend: 1 frontend: 1 max_connections: 10000 attributes: comment: 2 rows in set (0.00 sec) mysql> update mysql_users set transaction_persistent=1 where username='root'; Query OK, 1 row affected (0.00 sec) mysql> update mysql_users set transaction_persistent=1 where username='stnduser'; Query OK, 1 row affected (0.00 sec) mysql> load mysql users to runtime; Query OK, 0 rows affected (0.01 sec) mysql> save mysql users to disk; Query OK, 0 rows affected (0.02 sec) # 测试连接终端,分别使用root用户和stnduser用户测试下它们是否能路由到默认的hostgroup_id=1(它是一个写组)读、写数据。 [root@Mysql-Proxy01 ~]# mysql -uroot -pPassword@01 -P6033 -h127.0.0.1 -e "select @@server_id"; # 这是通过转发端口6033连接的,连接的是转发到后端真正的数据库! +-------------+ | @@server_id | +-------------+ | 37 | +-------------+ [root@Mysql-Proxy01 ~]# mysql -uroot -pPassword@01 -P6033 -h127.0.0.1 -e "create database proxy_db"; # 远程Mysql-Master01并创建proxy_db [root@Mysql-Proxy01 ~]# mysql -uroot -pPassword@01 -P6033 -h127.0.0.1 -e "show databases like 'proxy_db'"; +---------------------+ | Database (proxy_db) | +---------------------+ | proxy_db | +---------------------+ [root@Mysql-Proxy01 ~]# mysql -ustnduser -pPassword@01 -P6033 -h127.0.0.1 -e "show databases like 'proxy_db'"; +---------------------+ | Database (proxy_db) | +---------------------+ | proxy_db | +---------------------+ # Proxy查询规则 mysql> show tables from stats; +--------------------------------------+ | tables | +--------------------------------------+ | global_variables | | stats_memory_metrics | | stats_mysql_commands_counters | | stats_mysql_connection_pool | | stats_mysql_connection_pool_reset | | stats_mysql_errors | | stats_mysql_errors_reset | | stats_mysql_free_connections | | stats_mysql_global | | stats_mysql_gtid_executed | | stats_mysql_prepared_statements_info | | stats_mysql_processlist | | stats_mysql_query_digest | | stats_mysql_query_digest_reset | | stats_mysql_query_rules | | stats_mysql_users | | stats_proxysql_servers_checksums | | stats_proxysql_servers_metrics | | stats_proxysql_servers_status | +--------------------------------------+ 19 rows in set (0.00 sec) mysql> select * from stats.stats_mysql_connection_pool\G # 显示与 MySQL 后端以及连接和总体流量相关的信息 *************************** 1. row *************************** hostgroup: 1 srv_host: 172.16.70.37 srv_port: 3306 status: ONLINE ConnUsed: 0 ConnFree: 1 ConnOK: 1 ConnERR: 0 MaxConnUsed: 1 Queries: 11 Queries_GTID_sync: 0 Bytes_data_sent: 258 Bytes_data_recv: 156 Latency_us: 1496 *************************** 2. row *************************** hostgroup: 2 srv_host: 172.16.70.181 srv_port: 3306 status: ONLINE ConnUsed: 0 ConnFree: 0 ConnOK: 0 ConnERR: 0 MaxConnUsed: 0 Queries: 0 Queries_GTID_sync: 0 Bytes_data_sent: 0 Bytes_data_recv: 0 Latency_us: 1320 2 rows in set (0.01 sec) mysql> select * from stats_mysql_commands_counters where total_cnt\G # 返回有关执行的语句类型和执行时间分布的详细信息 *************************** 1. row *************************** Command: CREATE_DATABASE Total_Time_us: 6514 Total_cnt: 6 cnt_100us: 0 cnt_500us: 1 cnt_1ms: 2 cnt_5ms: 3 cnt_10ms: 0 cnt_50ms: 0 cnt_100ms: 0 cnt_500ms: 0 cnt_1s: 0 cnt_5s: 0 cnt_10s: 0 cnt_INFs: 0 *************************** 2. row *************************** Command: SELECT Total_Time_us: 2683 Total_cnt: 13 cnt_100us: 11 cnt_500us: 0 cnt_1ms: 1 cnt_5ms: 1 cnt_10ms: 0 cnt_50ms: 0 cnt_100ms: 0 cnt_500ms: 0 cnt_1s: 0 cnt_5s: 0 cnt_10s: 0 cnt_INFs: 0 *************************** 3. row *************************** Command: SHOW Total_Time_us: 3158 Total_cnt: 3 cnt_100us: 0 cnt_500us: 0 cnt_1ms: 2 cnt_5ms: 1 cnt_10ms: 0 cnt_50ms: 0 cnt_100ms: 0 cnt_500ms: 0 cnt_1s: 0 cnt_5s: 0 cnt_10s: 0 cnt_INFs: 0 3 rows in set (0.00 sec) # 插入两个规则,目的是将select语句分离到hostgroup_id=2的读组,但由于select语句中有一个特殊语句SELECT...FOR UPDATE它会申请写锁,所以应该路由到hostgroup_id=1的写组。 mysql> insert into mysql_query_rules(rule_id,active,match_digest,destination_hostgroup,apply) VALUES (1,1,'^SELECT.*FOR UPDATE$',1,1), (2,1,'^SELECT',2,1); Query OK, 2 rows affected (0.01 sec) 说明: rule_id 规则的id。规则是按照rule_id的顺序进行处理的。 active 只有该字段值为1的规则才会加载到runtime数据结构,所以只有这些规则才会被查询处理模块处理。 username match_digest 用户名筛选,当设置为非NULL值时,只有匹配的用户建立的连接发出的查询才会被匹配。 destination_hostgroup 将匹配到的查询路由到该主机组。但注意,如果用户的transaction_persistent=1(见mysql_users表), 且该用户建立的连接开启了一个事务,则这个事务内的所有语句都将路由到同一主机组,无视匹配规则。 apply 当设置为1后,当匹配到该规则后,将立即应用该规则,不会再评估其它的规则(注意:应用之后,将不会评估mysql_query_rules_fast_routing中的规则)。 mysql> load mysql query rules to runtime; Query OK, 0 rows affected (0.00 sec) mysql> save mysql query rules to disk; Query OK, 0 rows affected (0.01 sec) 注意: select ... for update规则的rule_id必须要小于普通的select规则的rule_id,因为ProxySQL是根据rule_id的顺序进行规则匹配的。 # 再次测试,读操作是否给hostgroup_id=2的读组 [root@Mysql-Proxy01 ~]# mysql -uroot -pPassword@01 -P6033 -h127.0.0.1 -e "select @@server_id" +-------------+ | @@server_id | +-------------+ | 181 | +-------------+ # 再看写操作 [root@Mysql-Proxy01 ~]# mysql -uroot -pPassword@01 -P6033 -h127.0.0.1 -e 'start transaction;select @@server_id;commit;select @@server_id;' +-------------+ | @@server_id | +-------------+ | 37 | +-------------+ +-------------+ | @@server_id | +-------------+ | 181 | +-------------+ # 测试读写分离 [root@Mysql-Proxy01 ~]# mysql -uroot -pPassword@01 -P6033 -h127.0.0.1 Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 26 Server version: 5.5.30 (ProxySQL) Copyright (c) 2000, 2021, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databses; +--------------------+ | Database | +--------------------+ | information_schema | | mydb01 | | mysql | | performance_schema | | proxy_db | | sys | +--------------------+ # 创建数据 mysql> create table proxy_tb(name varchar(20),age int(4)); Query OK, 0 rows affected (0.03 sec) mysql> insert into proxy_tb values('zhaojiu','20'); Query OK, 1 row affected (0.01 sec) # 在proxysql管理端查看读写分离 [root@Mysql-Proxy01 ~]# mysql -uadmin -padmin -P6032 -h127.0.0.1 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 27 Server version: 5.5.30 (ProxySQL Admin Module) Copyright (c) 2000, 2021, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> select * from stats_mysql_query_digest; +-----------+--------------------+----------+----------------+--------------------+---------------------------------------------------+------------+------------+------------+----------+----------+----------+-------------------+---------------+ | hostgroup | schemaname | username | client_address | digest | digest_text | count_star | first_seen | last_seen | sum_time | min_time | max_time | sum_rows_affected | sum_rows_sent | +-----------+--------------------+----------+----------------+--------------------+---------------------------------------------------+------------+------------+------------+----------+----------+----------+-------------------+---------------+ | 2 | proxy_db | root | | 0x4A754D38BB86DC46 | select * from proxy_tb | 1 | 1629875080 | 1629875080 | 689 | 689 | 689 | 0 | 4 | | 2 | proxy_db | root | | 0x57D3532A839C8743 | SELECT * FROM `proxy_tb` WHERE ?=? | 1 | 1629875054 | 1629875054 | 9386 | 9386 | 9386 | 0 | 0 | | 1 | information_schema | root | | 0xF439051B8ABC188E | insert into proxy_tb values(?,?) | 1 | 1629875034 | 1629875034 | 5281 | 5281 | 5281 | 0 | 0 | | 2 | information_schema | root | | 0x620B328FE9D6D71A | SELECT DATABASE() | 2 | 1629865271 | 1629875054 | 6089 | 2261 | 3828 | 0 | 2 | | 1 | proxy_db | root | | 0x59D85BA7DD54E405 | create table proxy_tb(name varchar(?),age int(?)) | 1 | 1629865334 | 1629865334 | 29689 | 29689 | 29689 | 0 | 0 | +-----------+--------------------+----------+----------------+--------------------+---------------------------------------------------+------------+------------+------------+----------+----------+----------+-------------------+---------------+ 5 rows in set (0.00 sec) # 从上述结果就可以看出读写分离配置是成功的,读请求是转发到2组,写请求转发到1组 说明: hostgroup:查询将要路由到的目标主机组。如果值为-1,则表示命中了查询缓存,直接从缓存取数据返回给客户端。 schemaname:当前正在执行的查询所在的schema名称。 username:MySQL客户端连接到ProxySQL使用的用户名。 digest:一个十六进制的hash值,唯一地代表除了参数值部分的查询语句。 digest_text:参数化后的SQL语句的文本。注意,如果重写了SQL语句,则这个字段是显示的是重写后的字段。换句话说,这个字段是真正路由到后端,被后端节点执行的语句。 count_star:该查询(参数相同、值不同)总共被执行的次数。 first_seen:unix格式的timestamp时间戳,表示该查询首次被ProxySQL路由出去的时间点。 last_seen:unix格式的timestamp时间戳,到目前为止,上一次该查询被ProxySQL路由出去的时间点。 sum_time:执行该类查询所花的总时间(单位微秒)。在想要找出程序中哪部分语句消耗时间最长的语句时非常有用,此外根据这个结果还能提供一个如何提升性能的良好开端。 min_time, max_time:执行该类查询的时间范围。min_time表示的是目前为止执行该类查询所花的最短时间,max_time则是目前为止,执行该类查询所花的最长时间,单位都是微秒。
至此,MySQL主从复制(GTID)+读写分离的环境部署已经实现!