5

I tried to create a glue crawler which crawls a redshift table.The glue crawler executes successfully and creates an external table.But when I look at the metadata of the table I found "Input format","Output format","Serde name" and "Serde serialization lib" as null.Hence when I am trying to use the crawler table to read the data from Athena or spark I am getting exception.Below is the exception what I am facing when I am using spark to read the table.

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch table redshift_table. StorageDescriptor#InputFormat cannot be null for table: redshift_table (Service: null; Status Code: 0; Error Code: null;Request ID: null)

Below is the screenshot for glue crawler table properties.

Glue Crawler table properties

Kindly help me for the above issue.

trp
  • 384
  • 1
  • 5
  • 15
  • I dont think you can use athena against redshift. also see https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-redshift.html – Jon Scott May 15 '19 at 10:36
  • Its okay if I cant use in Athena.But I think I should be able to use in Spark. Eventhough when I use in Spark I am getting the exception. – trp May 15 '19 at 11:05
  • @trp - How did you fix this? – user3267989 Jul 27 '20 at 22:40

0 Answers0