I have a numpy array in pyspark and I would like to convert this to a DataFrame so I can write it out as a csv to view it.
I read the data in initially from a DataFrame however I had to convert to an array in order to use numpy.random.normal()
. Now I want to convert the data back so I can write it out as a csv to view it.
I have tried the following directly on the array
zarr.write.csv("/mylocation/inHDFS")
however I get the following error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'numpy.ndarray' object has no attribute 'write'
Any ideas?