1 2 3 4 5 6 7 8 9 10 11 12 13 | ERROR mapreduce.ExportJobBase: Export job failed! 20/11/24 16:47:36 ERROR tool.ExportTool: Error during export: Export job failed! at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445) at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931) at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80) at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | [huang@bigdata03 sqoop-1.4.7]$ bin/sqoop export --connect jdbc:mysql://bigdata03:3306/company -root --password 123456 --table users_sum_money --num-mappers 1 --export-dir /user/hive/warehousm_money --input-fields-terminated-by "\001" Warning: /opt/module/sqoop-1.4.7/bin/../../hbase does not exist! HBase imports will fail. Please set $HBASE_HOME to the root of your HBase installation. Warning: /opt/module/sqoop-1.4.7/bin/../../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /opt/module/sqoop-1.4.7/bin/../../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 20/11/24 16:46:34 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7 20/11/24 16:46:34 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecurr using -P instead. 20/11/24 16:46:34 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 20/11/24 16:46:34 INFO tool.CodeGenTool: Beginning code generation 20/11/24 16:46:35 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `users_sum_t LIMIT 1 20/11/24 16:46:35 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `users_sum_t LIMIT 1 20/11/24 16:46:35 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/module/hadoop-2.7.2 注: /tmp/sqoop-huang/compile/9b5ca81a5b66ecf4d25bae1169847043/users_sum_money.java使用或覆盖了已I。 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。 20/11/24 16:46:37 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-huang/compile/9b5ca4d25bae1169847043/users_sum_money.jar 20/11/24 16:46:37 INFO mapreduce.ExportJobBase: Beginning export of users_sum_money 20/11/24 16:46:37 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapredr 20/11/24 16:46:40 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is . Instead, use mapreduce.reduce.speculative 20/11/24 16:46:40 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is depnstead, use mapreduce.map.speculative 20/11/24 16:46:40 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use job.maps 20/11/24 16:46:40 INFO client.RMProxy: Connecting to ResourceManager at bigdata02/192.168.17.10 20/11/24 16:46:45 INFO input.FileInputFormat: Total input paths to process : 1 20/11/24 16:46:45 INFO input.FileInputFormat: Total input paths to process : 1 20/11/24 16:46:45 INFO mapreduce.JobSubmitter: number of splits:1 20/11/24 16:46:45 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is depnstead, use mapreduce.map.speculative 20/11/24 16:46:45 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1606196215606_000 20/11/24 16:46:46 INFO impl.YarnClientImpl: Submitted application application_1606196215606_000 20/11/24 16:46:46 INFO mapreduce.Job: The url to track the job: http://bigdata02:8088/proxy/app606196215606_0007/ 20/11/24 16:46:46 INFO mapreduce.Job: Running job: job_1606196215606_0007 20/11/24 16:47:25 INFO mapreduce.Job: Job job_1606196215606_0007 running in uber mode : false 20/11/24 16:47:25 INFO mapreduce.Job: map 0% reduce 0% 20/11/24 16:47:34 INFO mapreduce.Job: map 100% reduce 0% 20/11/24 16:47:35 INFO mapreduce.Job: Job job_1606196215606_0007 failed with state FAILED due tiled task_1606196215606_0007_m_000000 Job failed as tasks failed. failedMaps:1 failedReduces:0 20/11/24 16:47:36 INFO mapreduce.Job: Counters: 8 Job Counters Failed map tasks=1 Launched map tasks=1 Data-local map tasks=1 Total time spent by all maps in occupied slots (ms)=6332 Total time spent by all reduces in occupied slots (ms)=0 Total time spent by all map tasks (ms)=6332 Total vcore-milliseconds taken by all map tasks=6332 Total megabyte-milliseconds taken by all map tasks=6483968 20/11/24 16:47:36 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apacmapreduce.FileSystemCounter instead 20/11/24 16:47:36 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 55.3161 seconds (0 bytes 20/11/24 16:47:36 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is depre org.apache.hadoop.mapreduce.TaskCounter instead 20/11/24 16:47:36 INFO mapreduce.ExportJobBase: Exported 0 records. 20/11/24 16:47:36 ERROR mapreduce.ExportJobBase: Export job failed! 20/11/24 16:47:36 ERROR tool.ExportTool: Error during export: Export job failed! at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445) at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931) at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80) at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) |
原因是数据中有中文,而数据库表编码不支持
解决方案如下:
将表的数据导出,删除表后重新创建表,并指定编码DEFAULT CHARSET=utf8