Pig permission denied while hdfs file is readable

Question

Running the following commands is sucessful

hadoop fs -ls /path/
hadoop fs -cat /path/.pig_schema

And all the files in that dir has a -rwxr-xr-x permission

However, in the pig console, when running:

A = LOAD '/path/' USING PigStorage();
B = LIMIT A 5;
DUMP B;

Encounters a permission error

2015-08-27 08:47:59,734 [main] ERROR org.apache.pig.tools.grunt.Grunt - You don't have permission to perform the operation. Error from the server: Permission denied
2015-08-27 08:47:59,735 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2017: Internal error creating job configuration.

Any idea why ?

EDIT 1: Added error log

================================================================================ Pig Stack Trace --------------- ERROR 2017: Internal error creating job configuration.

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias B at org.apache.pig.PigServer.openIterator(PigServer.java:857) at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:746) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:196) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:171) at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69) at org.apache.pig.Main.run(Main.java:543) at org.apache.pig.Main.main(Main.java:157) Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias B at org.apache.pig.PigServer.storeEx(PigServer.java:956) at org.apache.pig.PigServer.store(PigServer.java:919) at org.apache.pig.PigServer.openIterator(PigServer.java:832) ... 7 more Caused by: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException: ERROR 2017: Internal error creating job configuration. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:874) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:297) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:177) at org.apache.pig.PigServer.launchPlan(PigServer.java:1285) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1270) at org.apache.pig.PigServer.storeEx(PigServer.java:952) ... 9 more Caused by: java.io.IOException: Permission denied at java.io.UnixFileSystem.createFileExclusively(Native Method) at java.io.File.createTempFile(File.java:1879) at java.io.File.createTempFile(File.java:1923) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:538) ... 14 more

What happens if you try a `hadoop fs -ls /` and `hadoop fs -ls /tmp`? — Balduz, Aug 27 '15 at 09:07
`/tmp` is `drwxrwxrwt`, and most of its content is `drwx------` — Uri Goren, Aug 27 '15 at 09:10
I run pig with `pig -stop_on_failure -Dmapreduce.job.acl-view-job=* -Dpig.temp.dir=/tmp` — Uri Goren, Aug 27 '15 at 09:14
make sure you are in mapreduce mode. pig -x mapreduce or pig — Kishore, Aug 27 '15 at 09:19
There are a lot of dirs there, not all of them I can share publicly, do you have any specific directory in mind you think might be relevant to the issue ? — Uri Goren, Aug 27 '15 at 09:34
My question is if you have any permissions issue when listing the root directory, no need to share the contents of the command itself. — Balduz, Aug 27 '15 at 10:24
Can you make this work from another location, or by loading data in from an ORC DB for example? -- For people who found this post when looking for [ERROR 1066: Unable to open iterator for alias](http://stackoverflow.com/questions/34495085/error-1066-unable-to-open-iterator-for-alias-in-pig-generic-solution) here is a [generic solution](http://stackoverflow.com/a/34495086/983722). — Dennis Jaheruddin, Dec 28 '15 at 15:16

Pig permission denied while hdfs file is readable

0 Answers0