C/C++教程

在HiveSQL执行过程中,报错Exception: Too many counters: 121 max=120

本文主要是介绍在HiveSQL执行过程中,报错Exception: Too many counters: 121 max=120,对大家解决编程问题具有一定的参考价值,需要的程序猿们随着小编来一起学习吧!

报错信息如下:

INFO  : Exception: Too many counters: 121 max=120
at org.apache.tez.common.counters.Limits.checkCounters(Limits.java:86)
at org.apache.tez.common.counters.Limits.incrCounters(Limits.java:93)
at org.apache.tez.common.counters.AbstractCounterGroup.addCounter(AbstractCounterGroup.java:76)
at org.apache.tez.common.counters.AbstractCounterGroup.addCounterImpl(AbstractCounterGroup.java:93)
at org.apache.tez.common.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:104)
at org.apache.tez.common.counters.AbstractCounterGroup.aggrAllCounters(AbstractCounterGroup.java:204)
at org.apache.tez.common.counters.AbstractCounters.aggrAllCounters(AbstractCounters.java:372)
at org.apache.tez.dag.app.dag.impl.DAGImpl.aggrTaskCounters(DAGImpl.java:773)
at org.apache.tez.dag.app.dag.impl.DAGImpl.getAllCounters(DAGImpl.java:722)
at org.apache.tez.dag.app.dag.impl.DAGImpl.getDAGStatus(DAGImpl.java:938)
at org.apache.tez.dag.app.dag.impl.DAGImpl.getDAGStatus(DAGImpl.java:981)

 

这是在执行Hive SQL的时候报的错,引擎使用的是Tez。

解决方法:

需要修改Hadoop的mapred-site.xml配置文件的mapreduce.job.counters.max参数,默认为120,需要设置成更大的值。

<property> 
    <name>mapreduce.job.counters.max</name> 
    <value>200</value> 
</property> 

 

调整后再次报错:Concurrency mode is disabled, not creating a lock manager

报错信息如下:

ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask. 
INFO  : Completed executing command(queryId=dingli_20200311174547_a0c682f0-af9e-4f13-97c3-f4f076e88621); Time taken: 63.102 seconds
INFO  : Concurrency mode is disabled, not creating a lock manager
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask. (state=08S01,code=1)

 

解决办法:修改hive的配置文件,取消该配置注释 exprot HADOOP_HEAPSIZE=1024

cd $HIVE_HOME/conf
cp hive-env.sh.template hive-env.sh
vim hive-env.sh
exprot HADOOP_HEAPSIZE=1024

 

这篇关于在HiveSQL执行过程中,报错Exception: Too many counters: 121 max=120的文章就介绍到这儿,希望我们推荐的文章对大家有所帮助,也希望大家多多支持为之网!