skar0 / fire-detection Goto Github PK

Forest fire detection using Convolutional Neural Networks

Python 100.00%

cnn cnn-keras cnn-classification forest-fire transfer-learning data-augmentation

fire-detection's Introduction

Forest fire detection using CNN

This project is an attempt to use convolutional neural networks (CNN) to detect the presence or the start of a forest fire in an image. The idea is that this model could be applied to detect a fire or a start of a fire from (aerial) surveillance footage of a forest. The model could be applied in real-time to low-framerate surveillance video (with fires not moving very fast, this assumption is somewhat sound) and give alert in case of fire.

A description of the project, along with examples of video annotation by our network is provided below.

Datasets

Provided dataset

Our network is trained on a provided dataset which contains images of three categories : 'fire', 'no fire', 'start fire' totalling around 6000 images. These images are mostly of forest or forest-like environments. Images labelled 'fire' contain visible flames, 'start fire' images contain smoke indicating the start of a fire. Finaly, images labelled 'no fire' are images taken in forests.

Augmenting the dataset

Our experiments showed that the network had trouble classifying 'start fire' images so we added images of this category to the dataset by extracting frames from videos showing the start of a fire. In order to train a network which generalizes well to new images, we used data augmentation functions provided by Keras to perform a series of random transformations (zooms, shifts, crops and rotations) on images before they are fed to the network.

Project structure

Our goal was to create a legible project which handles every aspect of CNN creation and training. The code is organized as follows :

├── launcher.py
├── transfer_learning.py
├── video_annotation.py
├── evaluate_model.py
├── transfer_learned_model.h5
├── setup/
│   ├── setup_datasets.py
│   └── naive_approach.py
├── notebooks/
│   ├── datasets_analysis.ipynb
│   └── performance_analysis.ipynb
├── custom_model/
│   ├── cladoh.py
│   ├── model_train_test.py
│   └── model_train_tester.py
├── video_examples/
│   ├── video_0.gif
│   ├── video_1.gif
│   ├── video_2.gif
│   └── video_3.gif

The datasets can be setup using functions defines in setup_datasets.py. The model we used which performs transfer learning from InceptionV3 is defined in transfer_learning.py, this module contains a function that defines a batch generator which performs data augmentation. The training process is also handled in this file, with the possibility of freezing layers and adapting the learning rate for fine-tuning. Modules video_annotation.py allows to annotate a video with predictions from our CNN and evaluate_model.py allows us to evaluate our model and mine difficult examples for the network.

Usage

Requirements:

The project was tested with the following versions of librairies:

  imageio==2.6.1
  imageio-ffmpeg==0.3.0
  Keras==2.1.6
  matplotlib==2.2.3
  numpy==1.15.1
  opencv-contrib-python==3.4.0.12
  Pillow==5.2.0
  tensorflow==1.5.1

Launcher

The module launcher.py contains a command-line parser which allows to launch our project.

Training

DATASET is the path to the dataset. PROPORTION must be in [0, 1] is the proportion of the dataset to be used for training, the rest is kept for validation. FREEZE is a boolean on whether to freeze the inception layers in the network. EPOCHS and BATCH_SIZE can be specified, their default values are 10 and 32.

launcher.py train [-h] -data DATASET -prop PROPORTION -freeze FREEZE [-epochs EPOCHS] [-batch BATCH_SIZE]

Fine tuning

MODEL_PATH is the path to a pre-trained model, this must be a file containing wieghts + architecture. LEARNING_RATE can be specified, its default value is 0.001 (0.01 is used when training and is the default value of the optimizer we used).

launcher.py tune [-h] -model MODEL_PATH [-lr LEARNING_RATE] -data DATASET -prop PROPORTION -freeze FREEZE [-epochs EPOCHS] [-batch BATCH_SIZE]

Perform a prediction

launcher.py predict [-h] -path IMAGE_PATH -model MODEL_PATH

Extract difficult examples

Extracts images which are hard to classify by the model. When the network performs a prediction with a confidence level (probability) lower than EXTRACT_THRESHOLD for the correct class of an image, the path to this image is yielded.

launcher.py extract [-h] -data DATASET -model MODEL_PATH -threshold EXTRACT_THRESHOLD

Metrics on a test set

Yields the metrics of our model on a test set.

launcher.py test [-h] -data DATASET -model MODEL_PATH

Video annotation

The video given by INPUT_VIDEO_PATH is processed and prediction is performed on its frames, the annotated video is written to OUTPUT_VIDEO_PATH. A FREQ can be given so that only one out of every FREQ frames is extracted for prediction to speed up processing.

launcher.py video [-h] -in INPUT_VIDEO_PATH -out OUTPUT_VIDEO_PATH -model MODEL_PATH [-freq FREQ]

Results

Trained model file

Our trained model file containing the model architecture and trained weights is the file transfer_learned_model.h5 at the root of the project.

Performance

The performance of our model, measured by categorical loss and accuracy is the following:

On the provided test set :

transfer_learned_model.h5
100 samples
loss : 0.3496805104602239 | acc : 0.9108910891089109

On the whole dataset :

transfer_learned_model.h5
5953 samples
loss : 0.0360565472869205 | acc : 0.9914328909793382

From our experiments, it seems that 'fire' and 'no fire' images are always lassified with high accuracy. Images labelled 'start fire' are harder to classify for the network. This may be explained by the fact that 'fire' images may contain smoke and that 'start fire' images sometimes contain small flames.

Video examples

Examples of videos annotated by our model can be found below.

fire-detection's People

Contributors

Stargazers

Watchers

fire-detection's Issues

How to run this project?

git lfs, faild download pretrained model

git lfs faild download pretrained model .
here is faild log.

batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.

thanks.

error program launcher.py

import argparse
import numpy as np
from keras.models import load_model
from keras.preprocessing import image
from evaluate_model import evaluate_model, extract_hard_samples
from transfer_learning import train_simpler_inception_based_model
from video_annotation import video_fire_detection
from keras.applications.inception_v3 import preprocess_input as inception_preprocess_input

if name == 'main':

classes = ['fire', 'no_fire', 'start_fire']

parser = argparse.ArgumentParser(description='Convolutional neural network for forest fire detection',
                                 formatter_class=argparse.ArgumentDefaultsHelpFormatter)

subparsers = parser.add_subparsers(title='',
                                   description='Network can be trained on a provided dataset or predictions can be'
                                               'made using a pre-trained model. Models can also be evaluated.',
                                   help='', dest='mode')

subparsers.required = True


parser_train = subparsers.add_parser('train',
                                     help='Create and train the simpler InceptionV3-based model.',
                                     formatter_class=argparse.ArgumentDefaultsHelpFormatter)

parser_train.add_argument('-data',
                          type=str,
                          action='store',
                          dest='dataset',
                          help='Path to the dataset on which to train.',
                          default=argparse.SUPPRESS,
                          required=True)

parser_train.add_argument('-prop',
                          type=float,
                          action='store',
                          dest='proportion',
                          help='Proportion of the dataset to be used for training (the rest is for validation).',
                          default=argparse.SUPPRESS,
                          required=True)

parser_train.add_argument('-freeze',
                          type=bool,
                          action='store',
                          dest='freeze',
                          help='Whether to freeze every layer except the last fully connected ones.',
                          default=argparse.SUPPRESS,
                          required=True)

parser_train.add_argument('-epochs',
                          type=int,
                          action='store',
                          dest='epochs',
                          help='Number of epochs.',
                          default=10,
                          required=False)

parser_train.add_argument('-batch',
                          type=int,
                          action='store',
                          dest='batch_size',
                          help='Size of a batch.',
                          default=32,
                          required=False)

parser_tune = subparsers.add_parser('tune', help='Fine-tune a pre-trained Inception-V3-based model.',
                                    formatter_class=argparse.ArgumentDefaultsHelpFormatter)

parser_tune.add_argument('-model',
                         type=str,
                         action='store',
                         dest='model_path',
                         help='Path to the pre-trained model.',
                         default=argparse.SUPPRESS,
                         required=True)

parser_tune.add_argument('-lr',
                         type=float,
                         action='store',
                         dest='learning_rate',
                         help='Learning rate to be used for fine-tuning.',
                         default=0.001,
                         required=False)

parser_tune.add_argument('-data',
                         type=str,
                         action='store',
                         dest='dataset',
                         help='Path to the dataset on which to train.',
                         default=argparse.SUPPRESS,
                         required=True)

parser_tune.add_argument('-prop',
                         type=float,
                         action='store',
                         dest='proportion',
                         help='Proportion of the dataset to be used for training (the rest is for validation).',
                         default=argparse.SUPPRESS,
                         required=True)

parser_tune.add_argument('-freeze',
                         type=bool,
                         action='store',
                         dest='freeze',
                         help='Whether to freeze every layer except the last fully connected ones.',
                         default=argparse.SUPPRESS,
                         required=True)

parser_tune.add_argument('-epochs',
                         type=int,
                         action='store',
                         dest='epochs',
                         help='Number of epochs.',
                         default=10,
                         required=False)

parser_tune.add_argument('-batch',
                         type=int,
                         action='store',
                         dest='batch_size',
                         help='Size of a batch.',
                         default=32,
                         required=False)

parser_predict = subparsers.add_parser('predict',
                                       help='Perform prediction on a provided picture.')

parser_predict.add_argument('-path',
                            type=str,
                            action='store',
                            dest='image_path',
                            help='Path to an image.',
                            default=argparse.SUPPRESS,
                            required=True)

parser_predict.add_argument('-model',
                            type=str,
                            action='store',
                            dest='model_path',
                            help='Path to a trained model.',
                            default=argparse.SUPPRESS,
                            required=True)

parser_video = subparsers.add_parser('video',
                                     help='Perform prediction on a video.',
                                     formatter_class=argparse.ArgumentDefaultsHelpFormatter)

parser_video.add_argument('-in',
                          type=str,
                          action='store',
                          dest='input_video_path',
                          help='Path to an mp4 video.',
                          default=argparse.SUPPRESS,
                          required=True)

parser_video.add_argument('-out',
                          type=str,
                          action='store',
                          dest='output_video_path',
                          help='Path to output the annotated mp4 video.',
                          default=argparse.SUPPRESS,
                          required=True)

parser_video.add_argument('-model',
                          type=str,
                          action='store',
                          dest='model_path',
                          help='Path to a trained model.',
                          default=argparse.SUPPRESS,
                          required=True)

parser_video.add_argument('-freq',
                          type=int,
                          action='store',
                          dest='freq',
                          help='Prediction is to be made every freq frames.',
                          default=12,
                          required=False)

parser_extract = subparsers.add_parser('extract',
                                       help='Extract hard examples from a dataset (samples classified with low '
                                            'confidence).')

parser_extract.add_argument('-data',
                            type=str,
                            action='store',
                            dest='dataset',
                            help='Path to a dataset.',
                            default=argparse.SUPPRESS,
                            required=True)

parser_extract.add_argument('-model',
                            type=str,
                            action='store',
                            dest='model_path',
                            help='Path to a trained model.',
                            default=argparse.SUPPRESS,
                            required=True)

parser_extract.add_argument('-threshold',
                            type=float,
                            action='store',
                            dest='extract_threshold',
                            help='Threshold for the hard examples.',
                            default=argparse.SUPPRESS,
                            required=True)

parser_test = subparsers.add_parser('test',
                                    help='Test a model on a test set of images.')

parser_test.add_argument('-data',
                         type=str,
                         action='store',
                         dest='dataset',
                         help='Path to a test set.',
                         default=argparse.SUPPRESS,
                         required=True)

parser_test.add_argument('-model',
                         type=str,
                         action='store',
                         dest='model_path',
                         help='Path to a trained model.',
                         default=argparse.SUPPRESS,
                         required=True)

parsed = parser.parse_args()

if parsed.mode == "train":

    train_simpler_inception_based_model(parsed.dataset,
                                        fine_tune_existing=None,
                                        freeze=parsed.freeze,
                                        learning_rate=0.001,
                                        percentage=parsed.proportion,
                                        nbr_epochs=parsed.epochs,
                                        batch_size=parsed.batch_size)

elif parsed.mode == "tune":

    train_simpler_inception_based_model(parsed.dataset,
                                        fine_tune_existing=parsed.model_path,
                                        freeze=parsed.freeze,
                                        learning_rate=parsed.learning_rate,
                                        percentage=parsed.proportion,
                                        nbr_epochs=parsed.epochs,
                                        batch_size=parsed.batch_size)

elif parsed.mode == "predict":

    model = load_model(parsed.model_path)

    img = image.load_img(parsed.image_path, target_size=(224, 224, 3))

    # processed image to feed the network
    processed_img = image.img_to_array(img)
    processed_img = np.expand_dims(processed_img, axis=0)
    processed_img = inception_preprocess_input(processed_img)

    # get prediction using the network
    predictions = model.predict(processed_img)[0]

    print(predictions)

elif parsed.mode == "video":

    video_fire_detection(parsed.input_video_path,
                         parsed.output_video_path,
                         parsed.model_path,
                         inception_preprocess_input,
                         (224, 224),
                         parsed.freq)

elif parsed.mode == "extract":
    print(extract_hard_samples(parsed.model_path,
                               inception_preprocess_input,
                               parsed.dataset,
                               parsed.extract_threshold))

elif parsed.mode == "test":
    print(evaluate_model(parsed.model_path,
                         classes,
                         inception_preprocess_input,
                         parsed.dataset,
                               parsed.extract_threshold))