2

I am running a tensorflow job on google ai-platform. After the model is trained I want to save model.history to file. Since I need to write to a cloud storage bucket I use tf.io.write_file. I have tried the following: 

hist=mymodel.fit(training_dataset_input,epochs=num_epochs,steps_per_epoch=int(training_samples/batch_size),validation_data=validation_dataset_input,validation_steps=1,verbose=1,callbacks[mycallbacks,tensorboard_cb,csv_logger])

tf.io.write_file(gs://<nameofbucket>/<nameoffile>,hist.history)

I get he following error message:

ValueError: Attempt to convert a value ({'loss': [355.3313500958558, 313.9355358472616], 'mean_squared_error': [355.3312, 313.93536], 'find_rms': [14.733875, 13.815437], 'val_loss': [292.5752868652344, 270.8216857910156], 'val_mean_squared_error': [292.5753, 270.8217], 'val_find_rms': [13.506351, 12.935535]}) with an unsupported type (<class 'dict'>) to a Tensor.

It looks like hist.history is a dictionary with strings as keys and lists as values. How can I get hist.history in the right format so that it can be written to file using tf.io.write_file?

siamsot
  • 1,062
  • 1
  • 7
  • 16
Helga Holmestad
  • 107
  • 9
  • 1
    have you tried using `tf.io.Gfile.gfile.write()` or `pandas` ? Link: https://www.tensorflow.org/api_docs/python/tf/io/gfile/GFile – Adarsh Jul 29 '19 at 13:49

3 Answers3

3

You can try to convert the dictionary into a list using this and then save it (this can help).

If you want to visualize the saved data easily, you can maybe save it as csv (this can help).

Hope it can suits your problem!

Baptiste Pouthier
  • 503
  • 3
  • 22
2

Thanks for all the answers. I found that a combination of your answer worked for me. In order to write any string I want to a file and format it as I like I can use: 

with tf.io.gfile.GFile(gs://<bucket>/<filename>,"w") as file:
        file.write(<any string object>)

It also worked to do:

json.dumps(str(hist.history))

I needed to convert the whole history object to a string since it contains floats.

Helga Holmestad
  • 107
  • 9
1

The docs say that the arg 'contents' should be a Tensor of type string. scalar.

I could think of the following: If you only just want to store your history with the TF library and no more processing will be performed after, you could use json.dumps to get a string from your dict and save it.

On the other hand, you can Store objects to GCS Buckets via API with Python so there will be no need for extra processing.

Kevin Quinzel
  • 1,172
  • 1
  • 9
  • 17