MySql数据库采用主主复制、主从同步架构即两主两从。
编号 | 角色 | ip地址 | 版本 |
---|---|---|---|
1 | MySql-master1 | 192.16.1.109 | MySql8.0.25-1.el7 |
2 | MySql-slave1 | 192.16.1.160 | MySql8.0.25-1.el7 |
3 | MySql-master2 | 192.16.1.208 | MySql8.0.25-1.el7 |
4 | MySql-slave2 | 192.16.1.50 | MySql8.0.25-1.el7 |
5 | ShardingSphere-proxy1 | 192.16.1.7 | ShardingSphere-proxy5.0.0 |
6 | ShardingSphere-proxy2 | 192.16.1.8 | ShardingSphere-proxy5.0.0 |
7 | Keepalived+HA(Master) | 192.16.1.20 | Keepalived+HA |
8 | Keepalived+HA(Backup) | 192.16.2.21 | Keepalived+HA |
9 | VIP | 192.16.1.193 | 无 |
查看jdk版本、保证jdk的版本在1.8以上
java -version
dmesg|grep -i numa # vim /etc/default/grub 修改 GRUB_CMDLINE_LINUX="rhgb quiet" 添加 numa=off 修改后内容为 GRUB_CMDLINE_LINUX="rhgb quiet numa=off"
grub2-mkconfig -o /etc/grub2.cfg #重启操作系统 reboot #重新确认numa被禁用 dmesg|grep -i numa
sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config setenforce 0
#临时禁用,重启系统失效 systemctl stop firewalld #永久禁用 systemctl disable firewalld
vim /etc/security/limits.conf * soft nproc 65535 * hard nproc 65535 * soft nofile 65535 * hard nofile 65535 # vim /etc/systemd/system.conf DefaultLimitNOFILE=65536 DefaultLimitNPROC=65536
tzselect #一依次执行以下顺序 5-9-1-1>yes rm /etc/localtime ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime #重启服务器 reboot
官网下载mysql8 Arm版本rpm包mysql-8.0.25-1.el7.aarch64.rpm-bundle.tar
https://cdn.mysql.com/archives/mysql-8.0/mysql-8.0.25-1.el8.aarch64.rpm-bundle.tar
#创建mysql用户设置密码 useradd mysql passwd fqhOcqOK$61F
/home/mysql
tar -xvf mysql-8.0.25-1.el7.aarch64.rpm-bundle.tar
rpm -ivh mysql-community-common-8.0.25-1.el7.aarch64.rpm ----nodeps –force rpm -ivh mysql-community-libs-8.0.25-1.el7.aarch64.rpm ----nodeps –force rpm -ivh mysql-community-client-8.0.25-1.el7.aarch64.rpm ----nodeps –force rpm -ivh mysql-community-server-8.0.25-1.el7.aarch64.rpm ----nodeps –force
rpm -qa | grep mysql
mysqld --initialize;
chown mysql:mysql /var/lib/mysql -R;
systemctl start mysqld.service; systemctl enable mysqld;
cat /var/log/mysqld.log | grep password #记录默认密码
mysql -uroot -p
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY '密码';
create user 'root'@'%' identified with mysql_native_password by '123456'; grant all privileges on *.* to 'root'@'%' with grant option;
flush privileges;
修改my.cnf配置文件,vim /etc/my.cnf
[mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid #主服务器唯一ID (与 Master2,...... MasterN 的不同点) server-id=1 #启用二进制日志 log-bin=mysql-bin #从库的中继日志,主库日志写到中继日志,中继日志再重做到从库 # relay-log=myslql-relay-bin # binlog保留时间7天 expire_logs_days=7 # binlog 文件的大小 max_binlog_size=1G #设置logbin格式。取值:STATEMENT (默认),ROW,MIXED binlog_format=ROW # 该从库是否写入二进制日志。如果需要成为多主则可启用。只读可以不需要 log-slave-updates=1 # 该服务器自增列的初始值。(与 Master2,...... MasterN 的不同点) auto-increment-offset=1 # 该服务器自增列增量。其默认值是1, 取值范围是1 .. 65535 auto-increment-increment=2 # 设置不要复制的数据库(可设置多个) binlog-ignore-db=information_schema binlog-ignore-db=mysql binlog-ignore-db=performance_schema binlog-ignore-db=sys #设置需要复制的数据库(可选)。 # 如果要配置了此项,则 mysql 只复制下面指定的数据库。 # 如果不配置此项,则 mysql 默认复制所有的数据库(不包含 binlog-ignore-db 的数据库) #binlog-do-db=需要复制的主数据库1 #binlog-do-db=需要复制的主数据库2
修改my.cnf配置文件,vim /etc/my.cnf
[mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid #主服务器唯一ID (与 Master2,...... MasterN 的不同点) server-id=3 #启用二进制日志 log-bin=mysql-bin #从库的中继日志,主库日志写到中继日志,中继日志再重做到从库 # relay-log=myslql-relay-bin # binlog保留时间7天 expire_logs_days=7 # binlog 文件的大小 max_binlog_size=1G #设置logbin格式。取值:STATEMENT (默认),ROW,MIXED binlog_format=ROW # 该从库是否写入二进制日志。如果需要成为多主则可启用。只读可以不需要 log-slave-updates=1 # 该服务器自增列的初始值。(与 Master2,...... MasterN 的不同点) auto-increment-offset=2 # 该服务器自增列增量。其默认值是1, 取值范围是1 .. 65535 auto-increment-increment=2 # 设置不要复制的数据库(可设置多个) binlog-ignore-db=information_schema binlog-ignore-db=mysql binlog-ignore-db=performance_schema binlog-ignore-db=sys #设置需要复制的数据库(可选)。 # 如果要配置了此项,则 mysql 只复制下面指定的数据库。 # 如果不配置此项,则 mysql 默认复制所有的数据库(不包含 binlog-ignore-db 的数据库) #binlog-do-db=需要复制的主数据库1 #binlog-do-db=需要复制的主数据库2
[mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid #从服务器唯一ID server-id=2 #启用中继日志 relay-log=mysql-relay
[mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid #从服务器唯一ID server-id=4 #启用中继日志 relay-log=mysql-relay
systemctl restart mysqld.service
grant replication slave on *.* TO 'slave'@'%' identified by 'fqhOcqOK$61F';
如果报错,可以采用迂回的方式解决
create user slave identified BY 'fqhOcqOK$61F'; grant select, replication root, replication client on *.* TO 'slave'@'%'; grant all privileges on *.* TO 'slave'@'%'; flush privileges;
因为mysql8的加密方式与mysql5.7的不一样,在配置主从复制时容易出现Slave_IO_Running的状态不是YES,为了解决这个问题需要再执行一条命令
ALTER USER 'slave'@'%' IDENTIFIED WITH mysql_native_password BY 'fqhOcqOK$61F';
分别查看MySql-master1和MySql-master2的状态,记录下File和position的值,执行完此步骤后不要再操作主服务器MYSQL,防止主服务器状态值变化
show master status;
MySql-master1
MySql-master2
#复制主机的命令 change master to master_host='主机的IP地址', master_user='slave', master_password='123456', master_log_file='File的内容', master_log_pos=Position的内容;
change master to master_host='192.16.1.109', master_user='dog', master_password='123456', master_log_file='mysql-bin.000001', master_log_pos=1264;
change master to master_host='192.16.1.208', master_user='root', master_password='123456', master_log_file='mysql-bin.000001', master_log_pos=1264;
start slave;
查看Slave的启动状态
show slave status\G
正常情况下,Slave_IO_Running 、Slave_SQL_Running 两个参数都是Yes,则说明主从配置成功!如遇到这种情况就需要排查问题了,
1、停止从服务的复制功能
stop slave;
2、重置主从关系,重新配置主从
reset master;
因为是双主双从,所以这里MySql-master1配置MySql-master2,MySql-master2配置MySql-master1
change master to master_host='192.16.1.208', master_user='slave', master_password='密码', master_log_file='mysql-bin.000002',master_log_pos=156; ## Master1 的mysql-bin 相应参数
change master to master_host='192.16.1.109', master_user='slave', master_password='密码', master_log_file='mysql-bin.000002',master_log_pos=156; ## Master1 的mysql-bin 相应参数
start slave;
show slave status\G
在 MySql-master1主机新建库、新建表、 insert 记录, 然后在MySql-master2、MySql-slave1、MySql-slave2上查看复制 是否成功
官网下载 https://mirrors.bfsu.edu.cn/apache/shardingsphere/5.0.0/apache-shardingsphere-5.0.0-shardingsphere-proxy-bin.tar.gz 其他版本自行官网下载
将压缩包上传到指定的路径下解压
mkdir -P /home/shardingsphere tar -zxvf apache-shardingsphere-5.0.0-shardingsphere-proxy-bin.tar.gz -C /home/shardingsphere mv apache-shardingsphere-5.0.0 proxy
下载合适的驱动包,这里的驱动包是指与mysql数据库jdbc连接的jar包,官网下载:https://repo1.maven.org/maven2/mysql/mysql-connector-java/拷贝合适的驱动包到ShardingSphere-Proxy根目录的lib下(注:pgsql不需要引入)
/home/shardingsphere/proxy/lib/
<br />
# # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ###################################################################################################### # # If you want to configure governance, authorization and proxy properties, please refer to this file. # ###################################################################################################### scaling: blockQueueSize: 10000 workerThread: 40 clusterAutoSwitchAlgorithm: type: IDLE props: incremental-task-idle-minute-threshold: 30 dataConsistencyCheckAlgorithm: type: DEFAULT # #mode: # type: Cluster # repository: # type: ZooKeeper # props: # namespace: governance_ds # server-lists: localhost:2181 # retryIntervalMilliseconds: 500 # timeToLiveSeconds: 60 # maxRetries: 3 # operationTimeoutMilliseconds: 500 # overwrite: false # rules: - !AUTHORITY users: - root@%:passwd - sharding@:sharding provider: type: ALL_PRIVILEGES_PERMITTED - !TRANSACTION defaultType: XA providerType: Atomikos props: max-connections-size-per-query: 1 kernel-executor-size: 16 # Infinite by default. proxy-frontend-flush-threshold: 128 # The default value is 128. proxy-opentracing-enabled: false proxy-hint-enabled: false sql-show: false check-table-metadata-enabled: false show-process-list-enabled: false # Proxy backend query fetch size. A larger value may increase the memory usage of ShardingSphere Proxy. # The default value is -1, which means set the minimum value for different JDBC drivers. proxy-backend-query-fetch-size: -1 check-duplicate-table-enabled: false sql-comment-parse-enabled: false proxy-frontend-executor-size: 0 # Proxy frontend executor size. The default value is 0, which means let Netty decide. # Available options of proxy backend executor suitable: OLAP(default), OLTP. The OLTP option may reduce time cost of writing packets to client, but it may increase the latency of SQL execution # if client connections are more than proxy-frontend-netty-executor-size, especially executing slow SQL. proxy-backend-executor-suitable: OLAP proxy-frontend-max-connections: 0 # Less than or equal to 0 means no limitation. sql-federation-enabled: false
# # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ###################################################################################################### # # Here you can configure the rules for the proxy. # This example is configuration of replica-query rule. # ###################################################################################################### # #schemaName: readwrite_splitting_db # #dataSources: # primary_ds: # url: jdbc:postgresql://127.0.0.1:5432/demo_primary_ds # username: postgres # password: postgres # connectionTimeoutMilliseconds: 30000 # idleTimeoutMilliseconds: 60000 # maxLifetimeMilliseconds: 1800000 # maxPoolSize: 50 # minPoolSize: 1 # replica_ds_0: # url: jdbc:postgresql://127.0.0.1:5432/demo_replica_ds_0 # username: postgres # password: postgres # connectionTimeoutMilliseconds: 30000 # idleTimeoutMilliseconds: 60000 # maxLifetimeMilliseconds: 1800000 # maxPoolSize: 50 # minPoolSize: 1 # replica_ds_1: # url: jdbc:postgresql://127.0.0.1:5432/demo_replica_ds_1 # username: postgres # password: postgres # connectionTimeoutMilliseconds: 30000 # idleTimeoutMilliseconds: 60000 # maxLifetimeMilliseconds: 1800000 # maxPoolSize: 50 # minPoolSize: 1 # #rules: #- !READWRITE_SPLITTING # dataSources: # pr_ds: # writeDataSourceName: primary_ds # readDataSourceNames: # - replica_ds_0 # - replica_ds_1 ###################################################################################################### # # If you want to connect to MySQL, you should manually copy MySQL driver to lib directory. # ###################################################################################################### schemaName: nacos dataSources: write_ds: url: jdbc:mysql://192.16.1.109:3306/nacos?serverTimezone=UTC&useSSL=false username: root password: Csxpxc@1234! connectionTimeoutMilliseconds: 30000 idleTimeoutMilliseconds: 60000 maxLifetimeMilliseconds: 1800000 maxPoolSize: 50 minPoolSize: 1 read_ds_0: url: jdbc:mysql://192.16.1.160:3306/nacos?serverTimezone=UTC&useSSL=false username: root password: Csxpxc@1234! connectionTimeoutMilliseconds: 30000 idleTimeoutMilliseconds: 60000 maxLifetimeMilliseconds: 1800000 maxPoolSize: 50 minPoolSize: 1 read_ds_1: url: jdbc:mysql://192.16.1.50:3306/nacos?serverTimezone=UTC&useSSL=false username: root password: Csxpxc@1234! connectionTimeoutMilliseconds: 30000 idleTimeoutMilliseconds: 60000 maxLifetimeMilliseconds: 1800000 maxPoolSize: 50 minPoolSize: 1 rules: - !READWRITE_SPLITTING dataSources: pr_ds: writeDataSourceName: write_ds readDataSourceNames: - read_ds_0 - read_ds_1
vim start.sh 修改启动限制最小为328K
sh start.sh
mysql -uroot -P3307 -p
官网下载keepalived-1.4.5.tar.gz安装包,https://www.keepalived.org/software/keepalived-1.4.5.tar.gz
和haproxy-2.2.3.tar.gz ,https://src.fedoraproject.org/repo/pkgs/haproxy/
修改内核参数
#查看默认参数 [root@haproxy01 keepalived]# cat /proc/sys/net/ipv4/ip_nonlocal_bind 0 #修改参数 echo 1 > /proc/sys/net/ipv4/ip_nonlocal_bind sysctl -w net.ipv4.ip_nonlocal_bind=1 #永久生效 echo 'net.ipv4.ip_nonlocal_bind=1' >> /etc/sysctl.conf
将haproxy-2.2.3.tar.gz安装包上传到对应路径下解压、编译
cd /software tar -xvf haproxy-2.2.3.tar.gz cd /software/haproxy-2.2.3 make TARGET=linux2628 PREFIX=/usr/local/haproxy make install PREFIX=/usr/local/haproxy cp -rf /usr/local/haproxy/sbin/haproxy /usr/sbin/ cp examples/haproxy.init /etc/init.d/haproxy #赋权 chmod 755 /etc/init.d/haproxy #查看HAproxy的版本 haproxy -v
配置rsyslog
#创建目录 mkdir /var/log/haproxy #目录赋权 chmod a+w /var/log/haproxy
开启rsyslog记录HAproxy日志
vim /etc/rsyslog.conf #将如下两行得注释取消 $ModLoad imudp $UDPServerRun 514 #在该文件添加如下内容: # Save haproxy log local3.* /var/log/haproxy/haproxy.log
修改“/etc/sysconfig/rsyslog”文件,内容如下
vim /etc/sysconfig/rsyslog # Options for rsyslogd # Syslogd options are deprecated since rsyslog v3. # If you want to use them, switch to compatibility mode 2 by "-c 2" # See rsyslogd(8) for more details SYSLOGD_OPTIONS="-r -m 0 -c 2"
创建HAproxy运行用户
以root用户身份分别在所有集群节点上执行如下命令创建需要的组和用户:
groupadd -r haproxy useradd -g haproxy -M -s /sbin/nologin haproxy
配置HAproxy.cfg文件
以root用户身份分别在所有集群节点上编辑配置文件
vim /usr/local/haproxy/etc/haproxy.cfg配置haproxy.cfg文件
global #设置日志 chroot /usr/local/haproxy log 127.0.0.1 local3 info #用户与用户组 user haproxy group haproxy #定义每个haproxy进程的最大连接数 ,由于每个连接包括一个客户端和一个服务器端,所以单个进程的TCP会话最大数目将是该值的两倍。 maxconn 4096 # 以守护进程的方式运行 daemon defaults log global #日志中不记录空连接 option dontlognull # 定义连接后端服务器的失败重连次数,连接失败次数超过此值后将会将对应后端服务器标记为不可用 retries 3 option redispatch # 设置成功连接到一台服务器的最长等待时间,默认单位是毫秒 timeout connect 5000 # 设置连接客户端发送数据时的成功连接最长等待时间,默认单位是毫秒 timeout client 50000 # 设置服务器端回应客户度数据发送的最长等待时间,默认单位是毫秒 timeout server 50000 #统计页面 listen admin_stats bind 192.16.1.193:48800 mode http #采用http日志格式 option httplog #统计页面自动刷新时间 stats refresh 30s #统计页面url stats uri /admin_stats #统计页面密码框上提示文本 stats realm Haproxy Manager #统计页面用户名和密码设置 stats auth admin:admin #隐藏统计页面上HAProxy的版本信息 stats hide-version listen ShardingSphere_service # 绑定192.16.1.193:3307:3307端口访问端口ShardingSphere3307 bind 192.16.1.193:3307 # 定义为tcp模式 mode tcp #采用http日志格式 option tcplog # 开启对后端服务器的健康检测 option httpchk OPTIONS * HTTP/1.1\r\nHost:\ www # 设置haproxy的调度算法 balance roundrobin #根据调度分配到真实的后台地址,参数解释:port 48700:检测端口48700, inter 5s:5秒检测一次,rise 2:检测成功2次表示服务器可用,fall 3:检测失败3次后表示服务器不可用 server ShardingSphere1 192.16.1.7:3307 check port 48700 inter 5s rise 2 fall 3 server ShardingSphere2 192.16.1.8:3307 check port 48700 inter 5s rise 2 fall 3 #设置服务器端回应客户度数据发送的最长等待时间,默认单位是毫秒 timeout server 20000 5:9066 cookie 2 check port 48700 inter 5s rise 2 fall 3 timeout server 20000
启动HAproxy
# /usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/conf/haproxy.cfg
yum install -y pcre-devel openssl-devel popt-devel [root@master ~]# tar zxvf keepalived-1.4.5.tar.gz [root@master ~]# cd keepalived-1.4.5 [root@master ~]#./configure --prefix=/usr/local/keepalived make && make install
将编译好的文件拷贝到指定路径下
[root@master ~]# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/ [root@master ~]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/ [root@master ~]# mkdir /etc/keepalived/ [root@master ~]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/ [root@master ~]# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
Master节点
! Configuration File forkeepalived global_defs { script_user root enable_script_security notification_email { test@sina.com } notification_email_from admin@test.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id HAproxy #标识,双主相同 } vrrp_script chk_haproxy { script "/etc/keepalived/check_haproxy.sh" ## 检测haproxy状态的脚本路径 interval 2 ## 检测时间间隔 weight 2 ## 如果条件成立,权重+2 } vrrp_instance VI_1 { state MASTER #两台都设置BACKUP interface eth0 virtual_router_id 51 #主备相同 priority 100 #优先级,backup设置90 advert_int 1 nopreempt #不主动抢占资源,只在master这台优先级高的设置,backup不设置 authentication { auth_type PASS auth_pass 1111 } ## 将track_script块加入instance 配置块 track_script { chk_haproxy ## 检查HAProxy服务是否存活 } virtual_ipaddress { 192.16.1.193 } }
Backup节点
! Configuration File forkeepalived global_defs { script_user root enable_script_security notification_email { test@sina.com } notification_email_from admin@test.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id HAproxy #标识,双主相同 } vrrp_script chk_haproxy { script "/etc/keepalived/check_haproxy.sh" ## 检测haproxy状态的脚本路径 interval 2 ## 检测时间间隔 weight 2 ## 如果条件成立,权重+2 } vrrp_instance VI_1 { state BACKUP #两台都设置BACKUP interface eth0 virtual_router_id 51 #主备相同 priority 90 #优先级,backup设置90 advert_int 1 #不主动抢占资源,只在master这台优先级高的设置,backup不设置 authentication { auth_type PASS auth_pass 1111 } ## 将track_script块加入instance 配置块 track_script { chk_haproxy ## 检查HAProxy服务是否存活 } virtual_ipaddress { 192.16.1.193 } }
编辑端口探测脚本check_haproxy.sh
# vim /etc/keepalived/check_haproxy.sh //添加如下内容保存 #!/bin/bash START_HAPROXY="/etc/init.d/haproxy start" LOG_FILE="/var/log/keepalived-haproxy-state.log" if [ `ps -C haproxy --no-header |wc -l` -eq 0 ];then echo "`date "+%Y-%m-%d %H:%M:%S"` --- haproxy is down,next will restart haproxy" >> $LOG_FILE echo $START_HAPROXY >> $LOG_FILE $START_HAPROXY &>> $LOG_FILE sleep 2 if [ `ps -C haproxy --no-header |wc -l` -eq 0 ];then echo "start haproxy failed,will killall keepalived" >> $LOG_FILE killall keepalived else echo "`date "+%Y-%m-%d %H:%M:%S"` --- haproxy restart successful" >> $LOG_FILE fi fi
给脚本赋权
chmod +x /etc/keepalived/check_haproxy.sh
启动keepalived
systemctl start keepalived #查看keepalived运行状态 systemctl status keepalived chkconfig keepalived on //设置开机启动