sqoop 导入数据到 hive,报错 Container killed on request. Exit code is 143
报错信息:
19/04/10 09:49:01 INFO mapreduce.Job: map 0% reduce 0%
19/04/10 09:55:06 INFO mapreduce.Job: Task Id : attempt_1550821054018_1227_m_000000_0, Status : FAILED
[2019-04-10 09:55:04.761]Container [pid=56901,containerID=container_1550821054018_1227_01_000002] is running 8163328B beyond the 'PHYSICAL' memory limit. Current usage: 1.0 GB of 1 GB physical memory used; 2.6 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1550821054018_1227_01_000002 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 56901 56893 56901 56901 (bash) 3 9 9789440 293 /bin/bash -c /usr/java/jdk1.8.0_141-cloudera/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Djava.net.preferIPv4Stack=true -Xmx820m -Djava.io.tmpdir=/yarn/nm/usercache/impala/appcache/application_1550821054018_1227/container_1550821054018_1227_01_000002/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/yarn/container-logs/application_1550821054018_1227/container_1550821054018_1227_01_000002 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 10.252.96.54 36388 attempt_1550821054018_1227_m_000000_0 2 1>/yarn/container-logs/application_1550821054018_1227/container_1550821054018_1227_01_000002/stdout 2>/yarn/container-logs/application_1550821054018_1227/container_1550821054018_1227_01_000002/stderr
|- 59095 56901 56901 56901 (java) 16567 714 2741301248 263844 /usr/java/jdk1.8.0_141-cloudera/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Djava.net.preferIPv4Stack=true -Xmx820m -Djava.io.tmpdir=/yarn/nm/usercache/impala/appcache/application_1550821054018_1227/container_1550821054018_1227_01_000002/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/yarn/container-logs/application_1550821054018_1227/container_1550821054018_1227_01_000002 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 10.252.96.54 36388 attempt_1550821054018_1227_m_000000_0 2
[2019-04-10 09:55:04.774]Container killed on request. Exit code is 143
[2019-04-10 09:55:04.785]Container exited with a non-zero exit code 143.
原因:
内存分配不足,需要修改内存配置
解决方法:
修改CDH Yarn配置,将Map和Reduce任务的内存调大。
然后重启过期服务即可。