How to assign names to existing struct fields in inner row

Question

I have two seq objects. I am mergeing those two objects using

a.zip(b)

It gives me schema

 |    |-- element: struct (containsNull = true)
 |    |    |-- _1: string (nullable = true)
 |    |    |-- _2: double (nullable = false)

I want that schema to be

 |    |-- element: struct (containsNull = true)
 |    |    |-- name: string (nullable = true)
 |    |    |-- value: double (nullable = false)

How can I convert this?

It is an inner row to existing row

a and b are seq[String] and seq[Double] respectively – Chirag Jul 24 '17 at 02:14 — Chirag, Jul 24 '17 at 02:14

score 0 · Answer 1 · edited Jul 24 '17 at 02:22

0

chech the code please

a.zip(b).toDF("name","value")

edited Jul 24 '17 at 02:22

rogue-one

answered Jul 24 '17 at 01:31

Robin

It gives me exception java.lang.UnsupportedOperationException: Schema for type org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] is not supported at org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:733) at org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:671) at org.apache.spark.sql.functions$.udf(functions.scala:3088) – Chirag Jul 24 '17 at 02:16

1 Answers1