I have created simple dataprep workflow(Source File as CSV from GCS, simple transformation(Upper case conversion) & Target - load into BigQuery).
When i run this workflow job in DataPrep UI, I am getting error as:
Unable to rename output files from gs://test//temp/dax-tmp-2021-04-02_09_21_11-16351772716646701863-S07-0-d007cba17fe923f9/tmp-d007cba17fe92b44@DAX.ism to gs://test//temp/tmp-d007cba17fe92b44@.ism.,
I am having below IAM roles:
Dataprep User, Storage Admin, Storage Object Admin, Storage Object Creator, Viewer
Since i already have admin access in GCS, i am not sure why i am getting 'Unable to rename output files' error. But i am able to modify files are available in GCS using gsutil command.
Kindly advise whether this is access issue in DataPrep and how to solve this problem.
DataPrep Logs:
DataFlow Log:
