rdcolema / keras-image-classification Goto Github PK

Using Kaggle cats vs dogs dataset

Python 1.63% Jupyter Notebook 98.37%

keras-image-classification's Introduction

Keras Image Classification

Classifies an image as containing either a dog or a cat (using Kaggle's public dataset), but could easily be extended to other image classification problems.

To run these scripts/notebooks, you must have keras, numpy, scipy, and h5py installed, and enabling GPU acceleration is highly recommended if that's an option.

img_clf.py

After playing around with hyperparameters a bit, this reaches around 96-98% accuracy on the validation data, and when tested on Kaggle's hidden test data achieved a log loss score around 0.18.

Most of the code / strategy here was based on this Keras tutorial.

Pre-trained VGG16 model weights can be downloaded here.

The data directory structure I used was:

project
- data
  - train
    - dogs
    - cats
  - validation
    - dogs
    - cats
  - test
    - test

cats_n_dogs.ipynb:

This produced a slightly better score (.161 log loss on kaggle test set). The better score most likely comes from having larger images and ensembling a few models, despite the fact there's no image augmentation in the notebook.

Might run into memory errors because of the large image dimensions -- if so reducing the number of folds and saving the model weights rather than keeping the models in memory should do the trick. The notebook uses a slightly flatter directory structure, with the validation split happening after the images are loaded:

project
- data
  - train
    - dogs
    - cats
  - test
    - test

cats_n_dogs_BN.ipynb:

This produced the best score (0.069 loss without any ensembling). The notebook incorporates some of the techniques from Jeremy Howard's deep learning class , with the inclusion of batch normalization being the biggest factor. I also added extra layers of augmentation to the prediction script, which greatly improved performance.

Pre-trained model weights for VGG16 w/ batch normalization can be downloaded here.

The VGG16BN class is defined in vgg_bn.py, and the data directory structure used was:

project
- data
  - train
    - dogs
    - cats
  - validation
    - dogs
    - cats
  - test
    - test

keras-image-classification's People

Contributors

Stargazers

Watchers

keras-image-classification's Issues

How do you get the 'bottleneck_weights.h5' file?

Hi,

I am brand new to keras and machine learning in general. I want to create a keras model using the 'cats_n_dogs.ipynb' file, but I am not sure where to find the required 'bottleneck_weights.h5' file. Is there a link I can download the file from? I have already downloaded the 'vgg16_weights.h5' file.

Layer weight shape (3, 3, 200, 64) not compatible with provided weight shape (64, 3, 3, 3)

Hi Robert!

Have been trying to get to grips with Keras and stumbled across your handy repo!

I'm just trying to turn the handle on img_clf.py with the Theano backend and am having an issue with the save_bottleneck_features function (building the pretrained vgg16 model). I haven't made any modifications beyond updating the directories, so I was wondering if you had any idea why the weight shapes are so different?

I picked up the vgg16_weight.h5 file from the gdrive on the page you linked...

Any help would be much appreciated!

(Apologies if I have made some glaringly simple mistake)

Environment Specifications to run this code

Hey,
Could you please tell the environment specifications you used to run this code, because when I'm trying to run this with keras 2.0 and updated theano libraries, I am getting errors.

Layer weight shape not compatible with provided weight shape

I am the new one to learning using the keras and have problems about this code:

error:Layer weight shape (3, 3, 3, 64) not compatible with provided weight shape (64, 3, 3, 3)
i tried to change backend and "image_dim_ordering": "th", but those did not work

TypeError: Number of dimensions lower than expected

Hi there,
I get a type error when trying to run

predictions, filenames = vgg.test(test_path, nb_test_samples)

TypeError: Number of dimensions lower than expected Apply node that caused the error: Shape_i{0}(/lambda_1_input) Toposort index: 2 Inputs types: [TensorType(float32, 4D)] Inputs shapes: ['No shapes'] Inputs strides: ['No strides'] Inputs values: [None]

Can't load weight

Please refer to Question.
I am getting error in this line : model.layers[k].set_weights(weights)

Following is the error:
ValueError: Layer weight shape (3L, 3L, 3L, 64L) not compatible with provided weight shape (64, 3, 3, 3)

Please help

Error

I'm actually using AWS sagemaker as the platform and have the dadaset both in my local system and as well as in s3 bucket. But receiving the same error for both datasets! Receiving this the following error while trying to run the following code:

train_data, train_target, train_id = normalize_train_data()

Loading training images...
Loading dogs files (Index: 0)
Loading cats files (Index: 1)
Training data load time: 0.0 seconds

ValueError Traceback (most recent call last)
in ()
----> 1 train_data, train_target, train_id = normalize_train_data()

in normalize_train_data()
5 train_target = np.array(train_target, dtype=np.uint8)
6
----> 7 train_data = train_data.transpose((0, 3, 1, 2))
8
9 train_data = train_data.astype('float32')

ValueError: axes don't match array