2

In the context of creating and loading a .tfrecord file i encountered the following Problem:

Generating the dataset.tfrecord file

The Folder /Batch_manager/assets contains some *.tif Images that are used to generate a dataset.tfrecord file:

def _save_as_tfrecord(self, path, name):
    self.__filename = os.path.join(path, name + '.tfrecord')
    writer = tf.python_io.TFRecordWriter(self.__filename)
    print('Writing', self.__filename)
    for index, img in enumerate(self.load(get_iterator=True, n_images=1)):
        img = img[0]
        image_raw = img.tostring()
        rows = img.shape[0]
        cols = img.shape[1]
        try:
            depth = img.shape[2]
        except IndexError:
            depth = 1
        example = tf.train.Example(features=tf.train.Features(feature={
            'height': self._int64_feature(rows), 
            'width': self._int64_feature(cols), 
            'depth': self._int64_feature(depth), 
            'label': self._int64_feature(int(self.target[index])), 
            'image_raw': self._bytes_feature(image_raw)
                }))
        writer.write(example.SerializeToString())
    writer.close()

Reading from the dataset.tfrecord file

Next i try to read from this file using where path directs towards the dataset.tfrecord file:

def dataset_input_fn(self, path):
    dataset = tf.contrib.data.TFRecordDataset(path)

    def parser(record):
        keys_to_features = {
            "height": tf.FixedLenFeature((), tf.int64, default_value=""),
            "width": tf.FixedLenFeature((), tf.int64, default_value=""),
            "depth": tf.FixedLenFeature((), tf.int64, default_value=""),
            "label": tf.FixedLenFeature((), tf.int64, default_value=""),
            "image_raw": tf.FixedLenFeature((), tf.string, default_value=""),
        }
        print(record)
        features = tf.parse_single_example(record, features=keys_to_features)
        print(features)
        label = features['label']
        height = features['height']
        width = features['width']
        depth = features['depth']
        image = tf.decode_raw(features['image_raw'], tf.float32) 
        image = tf.reshape(image, [height, width, -1])
        label = tf.cast(features["label"], tf.int32)

        return {"image_raw": image, "height": height, "width": width, "depth":depth, "label":label}

    dataset = dataset.map(parser)
    dataset = dataset.shuffle(buffer_size=10000)
    dataset = dataset.batch(32)
    iterator = dataset.make_one_shot_iterator()

    # `features` is a dictionary in which each value is a batch of values for
    # that feature; `labels` is a batch of labels.
    features = iterator.get_next()

    return Features

Error message:

TypeError: Expected int64, got '' of type 'str' instead.

What is wrong with this Piece of code? I successfully validated that the dataset.tfrecord actually contains the correct Images and meta data!

Frederik Elischberger
  • 96
  • 1
  • 10
  • `self.load(...)` simply Returns an iterator that can be used to load on a per-Image Basis. I'am pretty sure that the issue is either because of the way i build the `example` variable and write it to dataset.tfrecord or the way it is parsed with `tf.contrib.data.TFRecordDataset(path)` and the `parser` function given to `.map(func)` – Frederik Elischberger Oct 06 '17 at 11:54

1 Answers1

0

The error happens because i copy and pasted this example which sets the values for all key-value pairs to an empty string, caused by default_value="". Removing that from all tf.FixedLenFeature fixed the issue.

Frederik Elischberger
  • 96
  • 1
  • 10