提问者:小点点

Maven Spark源代码在Ubuntu 20.04中构建失败


我正在尝试使用mvn build构建Spark 3.1.1源代码,如下所示:

./build/mvn-DskipTest干净包

但是构建失败没有给出任何正确的错误如下

杀死13456"${MVN_BIN}"-DzincPort=${ZINC_PORT}"$@"

任何帮助都非常感谢。

我的环境如下:OS: Ubuntu 20.04Java:JavaHotSpot(TM)64位服务器VM警告:忽略选项MaxPermSize=1G;在8.0java版本“1.8.0_271”Java(TM)SE运行时环境(构建1.8.0_271-b09)JavaHotSpot(TM)64位服务器VM(构建25.271-b09,混合模式)火花版本:火花3.1.1


共1个答案

匿名用户

我也遇到了同样的问题,经过排查,确认是Linux杀进程,原因是内存不足。

Maven日志:

[INFO] --- scala-maven-plugin:4.5.6:compile (scala-compile-first) @ spark-hive-thriftserver_2.12 ---
[INFO] Using incremental compilation using Mixed compile order
[INFO] Compiler bridge file: /home/tianshuang/.sbt/1.0/zinc/org.scala-sbt/org.scala-sbt-compiler-bridge_2.12-1.5.8-bin_2.12.15__52.0-1.5.8_20211211T222914.jar
[INFO] compiler plugin: BasicArtifact(com.github.ghik,silencer-plugin_2.12.15,1.7.6,null)
[INFO] compiling 27 Scala sources and 86 Java sources to /home/tianshuang/IdeaProjects/latest/spark/sql/hive-thriftserver/target/scala-2.12/classes ...
./build/mvn: line 185:  8772 Killed                  "${MVN_BIN}" "$@"

cmd:

dmesg -T | grep -i kill

输出:

[Wed Apr 13 22:07:44 2022] fsnotifier invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
[Wed Apr 13 22:07:44 2022]  oom_kill_process.cold+0xb/0x10
[Wed Apr 13 22:07:44 2022] [   1862]  1000  1862   114275       95   110592      117             0 gsd-rfkill
[Wed Apr 13 22:07:44 2022] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/user@1000.service,task=java,pid=8772,uid=1000
[Wed Apr 13 22:07:44 2022] Out of memory: Killed process 8772 (java) total-vm:15551840kB, anon-rss:4862120kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:10256kB oom_score_adj:0