Phoenix Build fail. Cannot find Symbol. Phoenix 4.8.2 Hbase-1.2 cdh 5.9.1

Question

I'm installing Apache Phoenix 4.8.2-Hbase-1.2 over my Cloudera's Hadoop and Hbase installation (i.e both are installed through cdh 5.9.1)

Followed these instructions to build Phoenix over it: Using Phoenix with Cloudera Hbase (installed from repo)

Now when i run sudo mvn install -DskipTests

I get this error in phoenix-core compilation:

[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Phoenix .................................... SUCCESS [5.569s]
[INFO] Phoenix Core ...................................... FAILURE [2:30.148s]
[INFO] Phoenix - Flume ................................... SKIPPED
[INFO] Phoenix - Pig ..................................... SKIPPED
[INFO] Phoenix Query Server Client ....................... SKIPPED
[INFO] Phoenix Query Server .............................. SKIPPED
[INFO] Phoenix - Pherf ................................... SKIPPED
[INFO] Phoenix - Spark ................................... SKIPPED
[INFO] Phoenix - Hive .................................... SKIPPED
[INFO] Phoenix Client .................................... SKIPPED
[INFO] Phoenix Server .................................... SKIPPED
[INFO] Phoenix Assembly .................................. SKIPPED
[INFO] Phoenix - Tracing Web Application ................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2:37.862s
[INFO] Finished at: Fri Jan 20 13:02:44 IST 2017
[INFO] Final Memory: 68M/714M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile) on project phoenix-core: Compilation failure: Compilation failure:
[ERROR] /opt/apache-phoenix-4.8.2-HBase-1.2-src-copy/phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixMRJobUtil.java:[222,29] cannot find symbol
[ERROR] symbol:   variable QUEUE_NAME
[ERROR] location: interface org.apache.hadoop.mapreduce.MRJobConfig
[ERROR] /opt/apache-phoenix-4.8.2-HBase-1.2-src-copy/phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixMRJobUtil.java:[226,32] cannot find symbol
[ERROR] symbol:   variable MAP_MEMORY_MB
[ERROR] location: interface org.apache.hadoop.mapreduce.MRJobConfig
[ERROR] /opt/apache-phoenix-4.8.2-HBase-1.2-src-copy/phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixMRJobUtil.java:[227,29] cannot find symbol
[ERROR] symbol:   variable MAP_JAVA_OPTS
[ERROR] location: interface org.apache.hadoop.mapreduce.MRJobConfig
[ERROR] /opt/apache-phoenix-4.8.2-HBase-1.2-src-copy/phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixMRJobUtil.java:[229,54] cannot find symbol
[ERROR] symbol:   variable QUEUE_NAME
[ERROR] location: interface org.apache.hadoop.mapreduce.MRJobConfig
[ERROR] /opt/apache-phoenix-4.8.2-HBase-1.2-src-copy/phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixMRJobUtil.java:[230,39] cannot find symbol
[ERROR] symbol:   variable MAP_MEMORY_MB
[ERROR] location: interface org.apache.hadoop.mapreduce.MRJobConfig
[ERROR] /opt/apache-phoenix-4.8.2-HBase-1.2-src-copy/phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixMRJobUtil.java:[231,39] cannot find symbol
[ERROR] symbol:   variable MAP_JAVA_OPTS
[ERROR] location: interface org.apache.hadoop.mapreduce.MRJobConfig
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :phoenix-core

Its not able to find QUEUE_NAME MAP_MEMORY_MB and MAP_JAVA_OPTS in org.apache.hadoop.mapreduce.MRJobConfig class. Doesn't cloudera hadoop 2.6 contain these variables?

Also only this public static void updateCapacityQueueInfo(Configuration conf) function in PhoenixMRJobUtil.java is using these variables and causing the compilation failure. This is the function providing info for Capacity scheduler. Is it safe to comment this function and its calls from the phoenix code, since I'm installing these for local testing only and not for production environment.

Otherwise what else could be the resolution? Aren't hadoop 2.6 and phoenix 4.8.2 compatible?

whitewalker · Accepted Answer · 2017-01-21T23:33:36.820

Finally built the Phoenix 4.8.2 successfully for cdh5.9.1 (Hbase 1.2.0 and Hadoop 2.6.0). These issues are due to incompatibility of Phoenix 4.8.2 with cdh5.9.1.

After trying my hands on changing the phoenix code to compile it successfully, I checked out phoenix 4.8Hbase-1.2-cdh5.8 branch from this repo in which most of the code changes have been done already to make it compatible:

https://github.com/chiastic-security/phoenix-for-cloudera/branches

Turns out this branch although for cdh5.8, worked for my cdh5.9.1 as well. Just changed the hbase and hadoop.two (including mr) versions in the pom.xml file and added this:

</repository>
    <repository>
  <id>conjars.org</id>
  <url>http://conjars.org/repo</url>
</repository>
<repository>
  <id>apache snapshot</id>
  <url>https://repository.apache.org/content/repositories/snapshots/</url>
  <snapshots>
    <enabled>true</enabled>
  </snapshots>
</repository>
<repository>
  <id>sonatype-nexus-snapshots</id>
  <name>Sonatype Nexus Snapshots</name>
  <url>https://oss.sonatype.org/content/repositories/snapshots</url>
  <snapshots>
    <enabled>true</enabled>
  </snapshots>
</repository>

Also replaced the version information for pig, jackson, slf4j, flume, spark with the ones mentioned in the original source's pom.xml here:

http://www-us.apache.org/dist/phoenix/apache-phoenix-4.8.2-HBase-1.2/src/

Phoenix Build fail. Cannot find Symbol. Phoenix 4.8.2 Hbase-1.2 cdh 5.9.1

1 Answers1