-1

I read online that you can save the contents of a dataframe to a CSV file, in a data lake, using a couple different methods. My dataframe is just fine, but I can't seem to save it to a CSV file. I am happy to put this CSV in the lake, or on my desktop. Either is totally fine.

Attempt 1:

df.write.csv("/data/home/csv")

Attempt 2:

df.coalesce(1).write.csv("/data/home/sample.csv")

Neither options work for me. With both, I am getting an error message that reads 

AttributeError: 'DataFrame' object has no attribute 'write'

Has anyone here actually gotten this to work? I am working in a Databricks environment. TIA.

ASH
  • 15,523
  • 6
  • 50
  • 116

1 Answers1

0

Use the 'to_csv' method of dataframe object to write the content into a csv file:

df.to_csv(r"filepathAndFilename.csv", index=False)

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_csv.html

san
  • 1,179
  • 6
  • 10
  • That looks like it should work, AFAIK, but I'm getting this error now: FileNotFoundError: [Errno 2] No such file or directory: 'dbfs/mnt/rawdata/2019/01/01/corp/AAA.csv' – ASH Oct 13 '19 at 01:43
  • @asher If the file is not already there, it would create the file. But if the directory information is wrong, it would throw error (which is what I am guessing in your case). Please make sure that you are providing proper path. – san Oct 13 '19 at 01:48
  • Thanks, but this doesn't seem to work in a Databricks environment. I'm on the server machine, and I'm trying to save the dataframe to a CSV in the data lake or save the dataframe to a CSV on my desktop, on the client machine. – ASH Oct 13 '19 at 12:44
  • I see this is for databricks. Let me know if these links help. I will remove my answers once you have acknowledged that you have seen this message: 1>https://forums.databricks.com/questions/14991/write-from-a-dataframe-to-a-csv-file-csv-file-is-b.html 2> https://docs.databricks.com/data/data-sources/read-csv.html#writing-files , 3> https://towardsdatascience.com/databricks-how-to-save-files-in-csv-on-your-local-computer-3d0c70e6a9ab – san Oct 13 '19 at 18:27