Failed to compile HDP hive 2.5.0.0 and 2.4.2.40 with the same error

Question

I failed to compile HDP hive because there isn't any jar including org.apache.hadoop.ipc.CallerContext. I tried to search this class from apache hadoop 2.7.3 and apache hive 2.1.0 source, but nothing was found. So, I want to know the details you compiled it. The compile information:

[INFO] ------------------------------------------------------------------------
[INFO] Building Hive Shims 0.23 1.2.1
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-shims-0.23 ---
[INFO] Deleting /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/target
[INFO] Deleting /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23 (includes = [datanucleus.log, derby.log], excludes = [])
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-no-snapshots) @ hive-shims-0.23 ---
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-shims-0.23 ---
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-shims-0.23 ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-shims-0.23 ---
[INFO] Executing tasks


main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-shims-0.23 ---
[INFO] Compiling 6 source files to /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/target/classes
[INFO] -------------------------------------------------------------
[WARNING] COMPILATION WARNING : 
[INFO] -------------------------------------------------------------
[WARNING] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java: /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java uses or overrides a deprecated API.
[WARNING] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java: Recompile with -Xlint:deprecation for details.
[WARNING] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java: /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java uses unchecked or unsafe operations.
[WARNING] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java: Recompile with -Xlint:unchecked for details.
[INFO] 4 warnings 
[INFO] -------------------------------------------------------------
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[70,29] cannot find symbol
  symbol:   class CallerContext
  location: package org.apache.hadoop.ipc
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1405,47] package CallerContext does not exist
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1405,5] cannot find symbol
  symbol:   variable CallerContext
  location: class org.apache.hadoop.hive.shims.Hadoop23Shims
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1420,9] cannot find symbol
  symbol:   variable CallerContext
  location: class org.apache.hadoop.hive.shims.Hadoop23Shims
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1423,12] cannot find symbol
  symbol:   variable CallerContext
  location: class org.apache.hadoop.hive.shims.Hadoop23Shims
[INFO] 5 errors 
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Hive ............................................... SUCCESS [  4.679 s]
[INFO] Hive Shims Common .................................. SUCCESS [  6.317 s]
[INFO] Hive Shims 0.20S ................................... SUCCESS [  1.946 s]
[INFO] Hive Shims 0.23 .................................... FAILURE [  8.511 s]
[INFO] Hive Shims Scheduler ............................... SKIPPED
[INFO] Hive Shims ......................................... SKIPPED
[INFO] Hive Common ........................................ SKIPPED
[INFO] Hive Serde ......................................... SKIPPED
[INFO] Hive Metastore ..................................... SKIPPED
[INFO] Hive Ant Utilities ................................. SKIPPED
[INFO] Spark Remote Client ................................ SKIPPED
[INFO] Hive Query Language ................................ SKIPPED
[INFO] Hive Service ....................................... SKIPPED
[INFO] Hive Accumulo Handler .............................. SKIPPED
[INFO] Hive JDBC .......................................... SKIPPED
[INFO] Hive Beeline ....................................... SKIPPED
[INFO] Hive CLI ........................................... SKIPPED
[INFO] Hive Contrib ....................................... SKIPPED
[INFO] Hive HBase Handler ................................. SKIPPED
[INFO] Hive HCatalog ...................................... SKIPPED
[INFO] Hive HCatalog Core ................................. SKIPPED
[INFO] Hive HCatalog Pig Adapter .......................... SKIPPED
[INFO] Hive HCatalog Server Extensions .................... SKIPPED
[INFO] Hive HCatalog Webhcat Java Client .................. SKIPPED
[INFO] Hive HCatalog Webhcat .............................. SKIPPED
[INFO] Hive HCatalog Streaming ............................ SKIPPED
[INFO] Hive HWI ........................................... SKIPPED
[INFO] Hive ODBC .......................................... SKIPPED
[INFO] Hive Shims Aggregator .............................. SKIPPED
[INFO] Hive TestUtils ..................................... SKIPPED
[INFO] Hive Packaging ..................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 22.817 s
[INFO] Finished at: 2016-12-10T12:51:57+08:00
[INFO] Final Memory: 49M/306M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project hive-shims-0.23: Compilation failure: Compilation failure:
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[70,29] cannot find symbol
[ERROR] symbol:   class CallerContext
[ERROR] location: package org.apache.hadoop.ipc
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1405,47] package CallerContext does not exist
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1405,5] cannot find symbol
[ERROR] symbol:   variable CallerContext
[ERROR] location: class org.apache.hadoop.hive.shims.Hadoop23Shims
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1420,9] cannot find symbol
[ERROR] symbol:   variable CallerContext
[ERROR] location: class org.apache.hadoop.hive.shims.Hadoop23Shims
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1423,12] cannot find symbol
[ERROR] symbol:   variable CallerContext
[ERROR] location: class org.apache.hadoop.hive.shims.Hadoop23Shims
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hive-shims-0.23

How do you compile it? What commands do you use? Where do you get the sources and so on. — Daniel, Dec 11 '16 at 00:05
Thanks for your help. Now I compiled it successfully. https://community.hortonworks.com/questions/70848/failed-to-compile-hdp-hive-2500-and-24240-with-the.html#answer-70850 — Lawrence, Dec 20 '16 at 01:50

Lawrence · Accepted Answer · 2016-12-20T02:07:19.147

0

I simply compiled hdp hive 2.5.0.0 by mvn clean install -Dmaven.test.skip=true.

After that I got this error.

Later I changed into "mvn clean install -Dmaven.test.skip=true -Phadoop-2 " and everything is ok.

By the way, I updated my pom.xml before I was going to compile.

<repository> <id>public</id> <url>http://nexus-private.hortonworks.com/nexus/content/groups/public </url> </repository>

edited Dec 20 '16 at 02:07

answered Dec 20 '16 at 02:01

Lawrence

1
4

Failed to compile HDP hive 2.5.0.0 and 2.4.2.40 with the same error

1 Answers1