1

I'm using cloudera CDH5 with Pig 0.13 ( tried Pig 0.12.1 as well ) and I'm building my own custom loadFunc and I have a simple script that just load using the new loadFunc.

When running in local mode everything works fine and the output is correct.

But when running in mapreduce mode ( still locally ), I get the error:

Unable to open iterator for alias output_hash
    at org.apache.pig.PigServer.openIterator(PigServer.java:880)
    at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
    at org.apache.pig.Main.run(Main.java:478)
    at org.apache.pig.Main.main(Main.java:156)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.io.IOException: Job terminated with anomalous status FAILED
    at org.apache.pig.PigServer.openIterator(PigServer.java:872)
    ... 12 more

I tried setting the properties verbose to true or traces to debug, but didn't didn't get useful info.

Any insights on how to resolve or debug this ?

nomier
  • 390
  • 1
  • 2
  • 12
  • According to the logging that I added to my loadFunc the instance of loadFunc was created and the setLocation function was called, but RecordReader was never created and next was never called – nomier Nov 20 '14 at 01:45
  • Could you provide the code for your load function? – alphaloop Nov 20 '14 at 03:55
  • For people who found this post when looking for [ERROR 1066: Unable to open iterator for alias](http://stackoverflow.com/questions/34495085/error-1066-unable-to-open-iterator-for-alias-in-pig-generic-solution) here is a [generic solution](http://stackoverflow.com/a/34495086/983722). – Dennis Jaheruddin Dec 28 '15 at 14:53

1 Answers1

2

Found the solution. The problem was that I was using maven to build the project and I was building the jar with dependencies. This caused the dependencies that has class names with the same class path to override each other (like FileSystem.java for hadoop-hdfs and hadoop-common and the solution was just to build the jar without including dependencies.

nomier
  • 390
  • 1
  • 2
  • 12