TensorFlow's `tf.data` module provides a functional API for building input pipelines, using the `tf.data.Dataset` and `tf.data.Iterator` classes.
Questions tagged [tensorflow-datasets]
1667 questions
0
votes
1 answer
Bucketing a tupled Tensorflow dataset where each tupled element has different shapes
I am trying to modify an existing tensorflow code. First, a 2d matrix of words is converted to a dataset from a geneartor and by map_strings_to_ints function and converted into vocab index. Then the following function is called.
dataset =…
rizwan
- 9
- 6
0
votes
0 answers
Feed multiple Inputs into Graph using data.dataset API
I am trying to feed data to CTC LSTM network using tensorflow am using the Dataset API to feed data to placeholders. For that I have created a
iterator.get_next() iter_val whose output is
(
Niteya Shah
- 1,931
- 10
- 20
0
votes
0 answers
Restoring a saved model and evaluating on a new Tensorflow Data object
I have this saved model and I want to restore it. After I restore, I want to evaluate it on a new dataset which I feeding with a Tensorflow Data input pipeline.
import tensorflow as tf
from tfwrappers.tf_dataset import…
ARAT
- 700
- 1
- 11
- 32
0
votes
1 answer
How do I verify images in a TensorFlow Dataset?
I am creating a tensor flow dataset of labels and images using the following code taken from TensorFlow documentation here.
# Reads an image from a file, decodes it into a dense tensor, and resizes it
# to a fixed shape.
def…
MJH
- 3
- 1
0
votes
1 answer
Data generated with Tensorflow Dataset.from_generator results in error when iterator.get_next() is called on it
I'm new to Tensorflow. I followed some online posts and wrote code to get data from a generator.
The code looks like this:
def gen(my_list_of_files):
for fl in my_list_of_files:
with open(fl) as f:
for line in f.readlines():
…
c3p0
- 105
- 2
- 11
0
votes
1 answer
How to use `tf.data.Dataset` in Tensorflow V2 with graphs?
I spot that functions, such as make_one_shot_iterator, have been deprecated in Tensorflow V2, and the suggestion is to use for ... in dataset, which requires eager execution enabled. I'm confused that without
these functions, how can we extract data…
Maybe
- 1,667
- 2
- 15
- 32
0
votes
0 answers
How to speed up simple CSV dataset
I'm trying to set up a tf.data.Dataset to stream larger-than-memory CSV files for training. I created the following benchmark to estimate processing throughput in mb/sec using a ~100mb CSV.
I'm processing in batches of 32. Ideally, I'd like to have…
Kevin
- 1
- 1
0
votes
1 answer
Assign label and id for images folder in python
I have currently worked on machine learning model and i used CNN model for it. My project is signature detection in which each folder contains users signature(10 signature per user). I want to assign label and id to folder. structure is as…
srs
- 13
- 5
0
votes
1 answer
Tensorflow dataset generator inverted colors
I have a problem with TF dataset generator. I do not why, but when I get picture from dataset by running it through session, it returns Tensors where colors are inverted. I tried to changed BGR to RGB, but this is not the problem.
It is partially…
Tomas Batrla
- 143
- 1
- 12
0
votes
1 answer
Fail to parse TFRecords while creating TF dataset
I'm trying to write a code to parse TFRecords and create TF dataset. I create TFRecords file from a list of images and be able to read it back and decode my image successfully. My code is based on example from this blog. But when I try to read my…
Valentin
- 976
- 2
- 11
- 24
0
votes
1 answer
Keras autoencoder with Tensorflow Dataset API and logging to Tensorboard
I have simple autoencoder in Keras, I want to use logging to tensorboard (thus I need passing validation data), and load the data from TFRecord using the Tensorflow Dataset API using prefetch.
I read some articles about it, but they either omitted…
Matěj Račinský
- 1,266
- 1
- 11
- 23
0
votes
1 answer
Tensorflow data pipeline: Slow with caching to disk - how to improve evaluation performance?
I've built a data pipeline. Pseudo code is as follows:
dataset ->
dataset = augment(dataset)
dataset = dataset.batch(35).prefetch(1)
dataset = set_from_generator(to_feed_dict(dataset)) # expensive op
dataset = Cache('/tmp', dataset)
dataset =…
Lukas
- 9
- 6
0
votes
1 answer
Change tf.dataset source at runtime in Tensorflow
I have two tf.datasets, one for validation and one for training.
Every now and again I want to switch the data source so that I can run the validation and check some accuracy measure on it.
This blog suggests to use placeholders and feed normal…
Andreas Storvik Strauman
- 1,490
- 10
- 26
0
votes
1 answer
predicting in tensor flow
After training my 'parameters' (w1,w2, weights of filters in Conv net), saving them as parameters=sess.run(parameters)
I take an image img=[1,64,64,3], and pass it to mypredict(x,parameters) function to predict but it gives error. The functions are…
liwei
- 99
- 2
- 11
0
votes
1 answer
Shuffling the dataset after re-initializing the iterator in tensorflow
I am using the tensorflow dataset api to feed data into the model. To the best of my knowledge I should shuffle the dataset to achieve the best performance from the model. However, since I am training for one epoch and then test for one epoch and so…
I. A
- 1,938
- 17
- 47