请记录并分享下列内容:
mysql 单机 192.168.237.137:22
OceanBase zone1 单副本 192.168.237.132
mysql> CREATE DATABASE IF NOT EXISTS RUNOOB DEFAULT CHARSET utf8 COLLATE utf8_general_ci; mysql> create table Student(Sno char(9) primary key,Sname char(20) not null,Ssex char(2),Sage smallint,Sdept char(20)); 添加student表数据 mysql>insert into Student(Sno,Sname,Ssex,Sage,Sdept)values('201215121','李勇','男','20','CS'); mysql>insert into Student(Sno,Sname,Ssex,Sage,Sdept)values('201215122','刘晨','女','19','CS'); mysql>insert into Student(Sno,Sname,Ssex,Sage,Sdept)values('201215123','王敏','女','18','MA'); mysql>insert into Student(Sno,Sname,Ssex,Sage,Sdept)values('201215125','张立','男','19','IS');
mysqldump -h 127.1 -uroot -P3306 -p -d mxdata > mx_ddl.sql
MySQL [mxdata] > source mx_ddl.sql
这个导出来的脚本有几个特征:
SQL_NOTES
,DEFINER
语句等。System Requirements
Linux
JDK(1.8以上,推荐1.8)
Python(推荐Python2.6.X)
Apache Maven 3.x (Compile DataX)
官方的要求,基本上可用。但建议:Linux使用CentOS 6.9,因为配套的Python版本即是2.6.6,只需要配置Maven就好。Maven的版本建议使用3.2.5。刚开始用的3.8.1报错太多…
$ git clone git@github.com:alibaba/DataX.git
1
使用git下载源码的方式,参考别的文档。BTW: 外网不稳定访问,多试。
$ cd {DataX_source_code_home}
$ mvn -U clean package assembly:assembly -Dmaven.test.skip=true
3.1,一定要记得配置国内镜像,修改setting.xml
<mirror> <id>nexus-aliyun</id> <mirrorOf>central</mirrorOf> <name>Nexus aliyun</name> <url>https://maven.aliyun.com/repository/central</url> </mirror>
[ERROR] Failed to execute goal on project hdfsreader: Could not resolve dependencies for project com.alibaba.datax:hdfsreader:jar:0.0.1-SNAPSHOT: Could not find artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde in nexus-aliyun (https://maven.aliyun.com/repository/central) -> [Help 1]
从提示上看,是hdfsreader的包缺失,相应的hdfswriter也应该是不行的。因为我用不到这两个组件,于是在pom.xml中注释掉。
还有一个 oscarwriter,也用不到,也在pom.xml中注释掉。
[ERROR] Failed to execute goal on project oscarwriter: Could not resolve dependencies for project com.alibaba.datax:oscarwriter:jar:0.0.1-SNAPSHOT: Could not find artifact com.oscar:oscar:jar:7.0.8 at specified path /opt/DataX/oscarwriter/src/main/lib/oscarJDBC.jar -> [Help 1]
[INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 05:43 min [INFO] Finished at: 2021-05-28T00:42:50+08:00 [INFO] Final Memory: 305M/492M [INFO] ------------------------------------------------------------------------
{ "job": { "setting": { "speed": { "channel": 4 }, "errorLimit": { "record": 0, "percentage": 0.1 } }, "content": [ { "reader": { "name": "mysqlreader", "parameter": { "username": "datax", "password": "root123", "column": [ "*" ], "connection": [ { "table": [ "lcxtest1" ], "jdbcUrl": ["jdbc:mysql://192.168.32.130:3306/lcxtest?useUnicode=true&characterEncoding=utf8&useSSL=false"] } ] } }, "writer": { "name": "oceanbasev10writer", "parameter": { "obWriteMode": "insert", "column": [ "*" ], "preSql": [ "truncate table lcxtest1" ], "connection": [ { "jdbcUrl": "||_dsc_ob10_dsc_||obcluster:test_tenant||_dsc_ob10_dsc_||jdbc:oceanbase://192.168.32.111:2883/lcxtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true", "table": [ "lcxtest1" ] } ], "username": "root", "password":"root123", "writerThreadCount":10, "batchSize": 1000, "memstoreThreshold": "0.9" } } } ] } }
注明:这里需要
mysql增加一个同步账号 datax 可自行更改
onceanbase 修改 租户 账号密码
启动datax 进行同步
./datax.py ../job/mysql2ob.json (自行命名)