-1

I have a hive table with a column struct. I need to run a query to collect some rows of that column into array.

The end result should be array of struct<....>

I have tried "collect_set", but it is giving me an error that it cannot be used for complex types. I need some kind of "group_concat" as in mySSQL.

Do you have any idea, how i can achieve this (preferably without using any udf/udaf)??

Thanks.

UPDATE: I have a temporary solution by using UDAF. But I am wondering if it is possible with built-in functions.

jww
  • 83,594
  • 69
  • 338
  • 732
maze2002
  • 1
  • 1

1 Answers1

0

Does this work?

SELECT collect_set(column_containing_struct.struct_field_name) FROM your_table;
-- Or some variant of this

This assumes a few things:

  • Your table contains a column named "column_containing_struct" that contains a struct.
  • The struct in that column contains a field named "struct_field_name".
  • What you are trying to do is produce an array of all unique values which occur in that field of the struct.

If you're not trying to deduplicate and you want everything that ever appears there, use collect_list instead.

rchang
  • 4,712
  • 1
  • 13
  • 24
  • Unfortunately, it doesn't work for me. At the end I need:array>. What your query provides array – maze2002 Dec 23 '14 at 19:30
  • @maze2002 Ah, I now understand your goal more clearly. I'm glad to see you've worked out at least a temporary solution. – rchang Dec 23 '14 at 19:51