Spark Scala how to process multiple columns in single loop

Question

Bellow rows are dataframe for Mango,Apple,Orange columns respectively

   [10,20,30]
   [100,2000,300]
   [1000,200,3000]

For the above dataframe: I need to get a summary like

{Mango: 1110; Apple:2220; Orange:3330 }

How do i do this with Single iteration ?

score 0 · Answer 1 · answered Feb 08 '18 at 09:49

If you have a simple dataframe as below

+-----+-----+------+
|Mango|Apple|Orange|
+-----+-----+------+
|10   |20   |30    |
|100  |200  |300   |
|1000 |2000 |3000  |
+-----+-----+------+

you can do something like below

df.select(sum("Mango").as("Mango"), sum("Apple").as("Apple"), sum("Orange").as("Orange")).toJSON.rdd.foreach(println)

which would give you output as

{"Mango":1110,"Apple":2220,"Orange":3330}

1 Answers1