0

Learnt from Jerry Kurata on Pluralsight, I'm trying to recognize birds:

my dataset structure is:

enter image description here

My model training code is:

import glob
import matplotlib.pyplot as plt

from keras import backend as K
import tensorflow as tf
with K.tf.device("/device:GPU:0"):
    config = tf.ConfigProto(intra_op_parallelism_threads=4,
           inter_op_parallelism_threads=4, allow_soft_placement=True,
           device_count = {'CPU' : 1, 'GPU' : 1})
    session = tf.Session(config=config)
    K.set_session(session)

from keras.callbacks import EarlyStopping
from keras.applications.inception_v3 import InceptionV3, preprocess_input
from keras.preprocessing.image import ImageDataGenerator
from keras.optimizers import SGD
from keras.models import Model
from keras.layers import Dense, GlobalAveragePooling2D
# "/device:GPU:0"
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'

def get_num_files(path):
    if not os.path.exists(path):
        return 0
    return sum([len(files) for r, d, files in os.walk(path)])

def get_num_subfolders(path):
    if not os.path.exists(path):
        return 0
    return sum([len(d) for r, d, files in os.walk(path)])

def create_img_generator():
    return ImageDataGenerator(
        preprocessing_function=preprocess_input,
        rotation_range=30,
        width_shift_range=0.2,
        height_shift_range=0.2,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True
    )

Image_width, Image_height = 299, 299
Training_Epochs = 1
Batch_Size = 32
Number_FC_Neurons = 1024

train_dir = '.../birds/train'
validate_dir = '.../birds/validation'

num_train_samples = get_num_files(train_dir)
num_classes = get_num_subfolders(train_dir)
num_validate_samples = get_num_files(validate_dir)

num_epoch = Training_Epochs
batch_size = Batch_Size

train_image_gen = create_img_generator()
test_image_gen = create_img_generator()

train_generator = train_image_gen.flow_from_directory(
    train_dir,
    target_size=(Image_width, Image_height),
    batch_size = batch_size,
    seed = 42
)

validation_generator = test_image_gen.flow_from_directory(
    validate_dir,
    target_size=(Image_width, Image_height),
    batch_size=batch_size,
    seed=42
)

Inceptionv3_model = InceptionV3(weights='imagenet', include_top=False)
print('Inception v3 model without last FC loaded')

x = Inceptionv3_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(Number_FC_Neurons, activation='relu')(x)
predictions = Dense(num_classes, activation='softmax')(x)

model = Model(inputs=Inceptionv3_model.input, outputs=predictions)
print(model.summary())

print('\nFine tuning existing model')

Layers_To_Freeze = 172
for layer in model.layers[:Layers_To_Freeze]:
    layer.trainable = False
for layer in model.layers[Layers_To_Freeze:]:
    layer.trainable = True

model.compile(optimizer=SGD(lr=0.0001, momentum=0.9), loss='binary_crossentropy', metrics=['accuracy'])

cbk_early_stopping = EarlyStopping(monitor='val_acc', mode='max')

history_transfer_learning = model.fit_generator(
    train_generator,
    steps_per_epoch = num_train_samples,
    epochs=num_epoch,
    validation_data=validation_generator,
    validation_steps = num_validate_samples,
    class_weight='auto',
    callbacks=[cbk_early_stopping]
)

model.save('incepv3_transfer.h5', overwrite=True, include_optimizer=True)

My detector is

from keras.models import load_model
from keras.optimizers import SGD
from keras.preprocessing import image
from keras.applications.inception_v3 import preprocess_input
import matplotlib.pyplot as plt
import numpy as np
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'

class Detector:
    def __init__(self, model_path):
        self.model = load_model(model_path)
        print('input shape')   # output is always (None, None, None, 3), this should be wrong
        print(self.model.layers[0].input_shape)
        # self.model.summary()
        # self.model.compile(loss='binary_crossentropy', optimizer=SGD(lr=0.0001, momentum=0.9), metrics=['accuracy'])

    def preprocess_input(self, x):
        y = np.copy(x)
        y /= 255.
        y -= 0.5
        y *= 2.
        return y

    def load_image(self, img_path, show=False):
        img = image.load_img(img_path, target_size=(299,299))
        img_tensor = image.img_to_array(img)                    # (height, width, channels)
        img_tensor = np.expand_dims(img, axis=0)         # (1, height, width, channels), add a dimension because the model expects this shape: (batch_size, height, width, channels)
        # img_tensor /= 255.                                      # imshow expects values in the range [0, 1]
        img_tensor = preprocess_input(img_tensor)

        if show:
            plt.imshow(img_tensor[0])
            plt.axis('off')
            plt.show()

        return img_tensor

    def detect(self, img_path):
        img = self.load_image(img_path, True)
        classes = self.model.predict(img)
        return classes

from this link

And here is how I use them to predict whether an image has a bird or not:

from keras.models import Model
from detector import Detector
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'

model_path = 'incepv3_transfer.h5'
detective = Detector(model_path)

bird_img = 'b1.jpeg'
classes = detective.detect(bird_img)

print(classes)


bird_img = 'dog1.jpg'
classes = detective.detect(bird_img)

print(classes)

the output is always:

[[1.]]

Franva
  • 5,410
  • 16
  • 62
  • 122
  • 1
    I'm not exactly sure what you want to do. Do you only want to tell if there is a bird on the image or not? In `flow_from_directory` you need one folder containing images per class, do you provide these? I think you need both, images of birds and images without birds. If you only train on a single class, the network will only learn to yield a 1 independent from its input. How did your loss during training look like? How did the validation loss look like? If there went something wrong regarding the provided classes, both should stay at 0 all the time. – Solvalou Dec 04 '19 at 22:46
  • hi @Solvalou I want to be able to tell if there is a bird in an image. I have updated my code to include my data set folder structure. I only want to detect birds, no need to detect cats or dogs. I feel that I need to provide a folder called non-birds under train and validation folders??? but as I use binary_crossentropy, so do I still need to provide non-birds images? in what structure? – Franva Dec 04 '19 at 23:20
  • hi @Solvalou your questions are helpful! I recalled that I saw the accuracy was always 1.00 and loss is 1.xxx e -7. – Franva Dec 05 '19 at 00:41
  • Yes, I actually think you need to provide images with non-birds. By doing so, you will get two output categories in the end. But it doesn't make too much sense since both outputs are then highly correlated. So what you could do to get a single output again is to wrap your image generator in another generator. Here, you get the next output of your input generator, which will be an image and the category it belongs to. Then you yield a tuple, consisting of the image and a newly encoded class again. So the new class will be 0 if there is no bird and 1 if there is a bird. – Solvalou Dec 05 '19 at 06:46
  • Since you then are only working with one output, you should change the activation in your final dense layer from softmax to sigmoid. This is because softmax is used for multiple classes and is a generalisation of sigmoid. I'm also not sure if this works, I'm just thinking aloud. I can maybe try it by myself if I find the time to do it. – Solvalou Dec 05 '19 at 06:51
  • @Solvalou thanks a lot. I will give it a try. but I have 110K images so it's going to be another 13hours waiting >_ – Franva Dec 05 '19 at 23:28
  • You're welcome. One simple method would be to reduce your training data so a single epoch would be faster. Of course, this won't give you the best results in the end, but you could at least tell if the network is learning at all if you take a look at the loss. You could also try to increase your learning rate for this test, as this will also yield faster results. – Solvalou Dec 06 '19 at 05:47
  • @Solvalou I think the easiest method is to increase classes from 1 class(bird) to at least 2 classes (bird + whatever), then I can use all the available classification method and code. btw, have you tried the sigmoid? – Franva Dec 06 '19 at 11:11

0 Answers0