Giter VIP home page Giter VIP logo

tiny-dnn's Introduction




Maintainers Wanted

The project may be abandoned since the maintainer(s) are just looking to move on. In the case anyone is interested in continuing the project, let us know so that we can discuss next steps.


Join the chat at https://gitter.im/tiny-dnn/users Docs License Coverage Status

tiny-dnn is a C++14 implementation of deep learning. It is suitable for deep learning on limited computational resource, embedded systems and IoT devices.

Linux/Mac OS Windows
Build Status Build status

Table of contents

Check out the documentation for more info.

What's New

Features

  • Reasonably fast, without GPU:
    • With TBB threading and SSE/AVX vectorization.
    • 98.8% accuracy on MNIST in 13 minutes training (@Core i7-3520M).
  • Portable & header-only:
    • Runs anywhere as long as you have a compiler which supports C++14.
    • Just include tiny_dnn.h and write your model in C++. There is nothing to install.
  • Easy to integrate with real applications:
    • No output to stdout/stderr.
    • A constant throughput (simple parallelization model, no garbage collection).
    • Works without throwing an exception.
    • Can import caffe's model.
  • Simply implemented:
    • A good library for learning neural networks.

Comparison with other libraries

Please see wiki page.

Supported networks

layer-types

  • core
    • fully connected
    • dropout
    • linear operation
    • zero padding
    • power
  • convolution
    • convolutional
    • average pooling
    • max pooling
    • deconvolutional
    • average unpooling
    • max unpooling
  • normalization
    • contrast normalization (only forward pass)
    • batch normalization
  • split/merge
    • concat
    • slice
    • elementwise-add

activation functions

  • tanh
  • asinh
  • sigmoid
  • softmax
  • softplus
  • softsign
  • rectified linear(relu)
  • leaky relu
  • identity
  • scaled tanh
  • exponential linear units(elu)
  • scaled exponential linear units (selu)

loss functions

  • cross-entropy
  • mean squared error
  • mean absolute error
  • mean absolute error with epsilon range

optimization algorithms

  • stochastic gradient descent (with/without L2 normalization)
  • momentum and Nesterov momentum
  • adagrad
  • rmsprop
  • adam
  • adamax

Dependencies

Nothing. All you need is a C++14 compiler (gcc 4.9+, clang 3.6+ or VS 2015+).

Build

tiny-dnn is header-only, so there's nothing to build. If you want to execute sample program or unit tests, you need to install cmake and type the following commands:

cmake . -DBUILD_EXAMPLES=ON
make

Then change to examples directory and run executable files.

If you would like to use IDE like Visual Studio or Xcode, you can also use cmake to generate corresponding files:

cmake . -G "Xcode"            # for Xcode users
cmake . -G "NMake Makefiles"  # for Windows Visual Studio users

Then open .sln file in visual studio and build(on windows/msvc), or type make command(on linux/mac/windows-mingw).

Some cmake options are available:

options description default additional requirements to use
USE_TBB Use Intel TBB for parallelization OFF1 Intel TBB
USE_OMP Use OpenMP for parallelization OFF1 OpenMP Compiler
USE_SSE Use Intel SSE instruction set ON Intel CPU which supports SSE
USE_AVX Use Intel AVX instruction set ON Intel CPU which supports AVX
USE_AVX2 Build tiny-dnn with AVX2 library support OFF Intel CPU which supports AVX2
USE_NNPACK Use NNPACK for convolution operation OFF Acceleration package for neural networks on multi-core CPUs
USE_OPENCL Enable/Disable OpenCL support (experimental) OFF The open standard for parallel programming of heterogeneous systems
USE_LIBDNN Use Greentea LibDNN for convolution operation with GPU via OpenCL (experimental) OFF An universal convolution implementation supporting CUDA and OpenCL
USE_SERIALIZER Enable model serialization ON2 -
USE_DOUBLE Use double precision computations instead of single precision OFF -
USE_ASAN Use Address Sanitizer OFF clang or gcc compiler
USE_IMAGE_API Enable Image API support ON -
USE_GEMMLOWP Enable gemmlowp support OFF -
BUILD_TESTS Build unit tests OFF3 -
BUILD_EXAMPLES Build example projects OFF -
BUILD_DOCS Build documentation OFF Doxygen
PROFILE Build unit tests OFF gprof

1 tiny-dnn use C++14 standard library for parallelization by default.

2 If you don't use serialization, you can switch off to speedup compilation time.

3 tiny-dnn uses Google Test as default framework to run unit tests. No pre-installation required, it's automatically downloaded during CMake configuration.

For example, type the following commands if you want to use Intel TBB and build tests:

cmake -DUSE_TBB=ON -DBUILD_TESTS=ON .

Customize configurations

You can edit include/config.h to customize default behavior.

Examples

Construct convolutional neural networks

#include "tiny_dnn/tiny_dnn.h"
using namespace tiny_dnn;
using namespace tiny_dnn::activation;
using namespace tiny_dnn::layers;

void construct_cnn() {
    using namespace tiny_dnn;

    network<sequential> net;

    // add layers
    net << conv(32, 32, 5, 1, 6) << tanh()  // in:32x32x1, 5x5conv, 6fmaps
        << ave_pool(28, 28, 6, 2) << tanh() // in:28x28x6, 2x2pooling
        << fc(14 * 14 * 6, 120) << tanh()   // in:14x14x6, out:120
        << fc(120, 10);                     // in:120,     out:10

    assert(net.in_data_size() == 32 * 32);
    assert(net.out_data_size() == 10);

    // load MNIST dataset
    std::vector<label_t> train_labels;
    std::vector<vec_t> train_images;

    parse_mnist_labels("train-labels.idx1-ubyte", &train_labels);
    parse_mnist_images("train-images.idx3-ubyte", &train_images, -1.0, 1.0, 2, 2);

    // declare optimization algorithm
    adagrad optimizer;

    // train (50-epoch, 30-minibatch)
    net.train<mse, adagrad>(optimizer, train_images, train_labels, 30, 50);

    // save
    net.save("net");

    // load
    // network<sequential> net2;
    // net2.load("net");
}

Construct multi-layer perceptron (mlp)

#include "tiny_dnn/tiny_dnn.h"
using namespace tiny_dnn;
using namespace tiny_dnn::activation;
using namespace tiny_dnn::layers;

void construct_mlp() {
    network<sequential> net;

    net << fc(32 * 32, 300) << sigmoid() << fc(300, 10);

    assert(net.in_data_size() == 32 * 32);
    assert(net.out_data_size() == 10);
}

Another way to construct mlp

#include "tiny_dnn/tiny_dnn.h"
using namespace tiny_dnn;
using namespace tiny_dnn::activation;

void construct_mlp() {
    auto mynet = make_mlp<tanh>({ 32 * 32, 300, 10 });

    assert(mynet.in_data_size() == 32 * 32);
    assert(mynet.out_data_size() == 10);
}

For more samples, read examples/main.cpp or MNIST example page.

Contributing

Since deep learning community is rapidly growing, we'd love to get contributions from you to accelerate tiny-dnn development! For a quick guide to contributing, take a look at the Contribution Documents.

References

[1] Y. Bengio, Practical Recommendations for Gradient-Based Training of Deep Architectures. arXiv:1206.5533v2, 2012

[2] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86, 2278-2324.

Other useful reference lists:

License

The BSD 3-Clause License

Gitter rooms

We have gitter rooms for discussing new features & QA. Feel free to join us!

developers https://gitter.im/tiny-dnn/developers
users https://gitter.im/tiny-dnn/users

tiny-dnn's People

Contributors

alessandro-gentilini avatar azsane avatar beru avatar bhack avatar cdmh avatar delightrun avatar edgarriba avatar festlv avatar h4kor avatar hzxie avatar jiaolong avatar jimbozhang avatar lakshayg avatar nishnik avatar nyanp avatar pansk avatar pliptor avatar prlz77 avatar randl avatar reunanen avatar rmsalinas avatar rolandpersson avatar srk97 avatar stereomatchingkiss avatar syoyo avatar teemperor avatar tohnosakuya avatar wangyida avatar yan796 avatar yumetodo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tiny-dnn's Issues

Control over TBB number of threads

Hi,
here is a line I added to get control over the number of threads created by TBB:
in util.h line 154
...

ifdef CNN_USE_TBB

static tbb::task_scheduler_init tbbScheduler(tbb::task_scheduler_init::deferred);
typedef tbb::blocked_range blocked_range;
typedef tbb::task_group task_group;
...

then, the you can do as in OpenCV void cv::setNumThreads( int threads )

ifdef CNN_USE_TBB

if(tbbScheduler.is_active()) tbbScheduler.terminate();
if(numThreads > 0) tbbScheduler.initialize(numThreads);

endif

Cheers

Obtaining a low accuracy with Tiny CNN

Hello.

I'm developing a model and I like trying things in MATLAB before, so I trained a model which works with this framework: https://github.com/rasmusbergpalm/DeepLearnToolbox

But when I'm trying to build it with Tiny-CNN, I get an accuracy of around 37% when instead using DeepLearnToolbox I get 98%.

I don't know what I'm doing bad.

Whad I do:

  • Create images to ubyte.
  • I use the 1st example:

void sample1_convnet(void) {
// construct LeNet-5 architecture
network<mse, gradient_descent_levenberg_marquardt> nn;

    // connection table [Y.Lecun, 1998 Table.1]
#define O true
#define X false
    static const bool connection [] = {
        O, X, X, X, O, O, O, X, X, O, O, O, O, X, O, O,
        O, O, X, X, X, O, O, O, X, X, O, O, O, O, X, O,
        O, O, O, X, X, X, O, O, O, X, X, O, X, O, O, O,
        X, O, O, O, X, X, O, O, O, O, X, X, O, X, O, O,
        X, X, O, O, O, X, X, O, O, O, O, X, O, O, X, O,
        X, X, X, O, O, O, X, X, O, O, O, O, X, O, O, O
    };
#undef O
#undef X

    nn << convolutional_layer<tan_h>(32, 32, 5, 1, 6) // 32x32 in, 5x5 kernel, 1-6 fmaps conv
       << average_pooling_layer<tan_h>(28, 28, 6, 2) // 28x28 in, 6 fmaps, 2x2 subsampling
       << convolutional_layer<tan_h>(14, 14, 5, 6, 16,
                                     connection_table(connection, 6, 16)) // with connection-table
       << average_pooling_layer<tan_h>(10, 10, 16, 2)
       << convolutional_layer<tan_h>(5, 5, 5, 16, 120)
       << fully_connected_layer<tan_h>(120, 10);

    std::cout << "load models..." << std::endl;

    // load MNIST dataset
    std::vector<label_t> train_labels, test_labels;
    std::vector<vec_t> train_images, test_images;

    parse_mnist_labels("../../data/vuittonoutput_train_labels.idx1", &train_labels);
    parse_mnist_images("../../data/vuittonoutput_train_images.idx3", &train_images);
    parse_mnist_labels("../../data/test_train_labels.idx1", &test_labels);
    parse_mnist_images("../../data/test_train_images.idx3", &test_images);

    std::cout << "start learning" << std::endl;

//    for (int i = 0; i < train_images.size(); i++) {
//        cout << endl << "image " << i << " size = " << train_images.at(i).size();
//    }

    boost::progress_display disp(train_images.size());
    boost::timer t;
    int minibatch_size = 2;

    nn.optimizer().alpha *= std::sqrt(minibatch_size);

    // create callback
    auto on_enumerate_epoch = [&](){
        std::cout << t.elapsed() << "s elapsed." << std::endl;

        tiny_cnn::result res = nn.test(test_images, test_labels);

        std::cout << nn.optimizer().alpha << "," << res.num_success << "/" << res.num_total << std::endl;

        nn.optimizer().alpha *= 0.90; // decay learning rate
        nn.optimizer().alpha = std::max(0.0000001, nn.optimizer().alpha);

        disp.restart(train_images.size());
        t.restart();
    };

    auto on_enumerate_minibatch = [&](){ 
        disp += minibatch_size; 

        // weight visualization in imdebug
        /*static int n = 0;    
        n+=minibatch_size;
        if (n >= 1000) {
            image img;
            C3.weight_to_image(img);
            imdebug("lum b=8 w=%d h=%d %p", img.width(), img.height(), &img.data()[0]);
            n = 0;
        }*/
    };

    // training
    nn.train(train_images, train_labels, minibatch_size, 100, on_enumerate_minibatch, on_enumerate_epoch);

    std::cout << "end training." << std::endl;

    // test and show results
    nn.test(test_images, test_labels).print_detail(std::cout);

    // save networks
    std::ofstream ofs("LeNet-weights");
    ofs << nn;
}

am I doing anything bad? I've tried to change some parameters but the result is almost the same.
Thank you very much in advance.

Is the back_propagation_2nd compute the approximate 2nd derivation?

I am currently add long_short_term_memory lstm_layer to tiny-cnn use for sequence labeling,by modifying the input_layer to recieve the data from fixed size window slide on 2d image.
I find the back_propagation_2nd function maybe not compute the 2nd derivation correctlly.
prev_delta2_[c] *= sqr(prev_h.df(prev_out[c]));
Is this compute the approximate 2nd derivation and let it >= 0?

Build fails with internal compiler error

./waf configure --BOOST_ROOT=/usr/include/boost
Setting top to : /tiny-cnn-master
Setting out to : /tiny-cnn-master/build
Checking for program g++,c++ : g++
Checking for program ar : /usr/bin/ar
'configure' finished successfully (0.034s)

./waf build
Waf: Entering directory /tiny-cnn-master/build' [1/2] cxx: src/main.cpp -> build/src/main.cpp.1.o In file included from /tiny-cnn-master/include/fully_connected_layer.h:28:0, from /tiny-cnn-master/include/network.h:39, from /tiny-cnn-master/include/tiny_cnn.h:31, from ../src/main.cpp:31: /tiny-cnn-master/include/layer.h: In lambda function: /tiny-cnn-master/include/layer.h:114:32: internal compiler error: Segmentation fault Please submit a full bug report, with preprocessed source if appropriate. See <file:///usr/share/doc/gcc-4.7/README.Bugs> for instructions. Preprocessed source stored into /tmp/ccXet206.out file, please attach this to your bugreport. Waf: Leaving directory/tiny-cnn-master/build'
Build failed

I am using gcc-4.7.2 on Debian Wheezy amd64 and boost-1.49.0.1.
Any idea why this fails?
Seems to complain about a lambda in layer.h

Use tiny-cnn in Ubuntu eclipse

can I only use tiny-cnn package as .h files folder. Then other cpp file in the project just use include "_.h".But i have some trouble with this idea.Almost caused by "_ can not resolve"

Run-time

For my notebook, core i5. It takes more than several hours to run through the 1st epoch, and accuracy is very low. I didn't revise any parameters but just download and run.
What is wrong?

Provide a tutorial of how to convert from my images dataset to idx3-ubyte file ?

Hello.

I've read the page of MNIST: http://yann.lecun.com/exdb/mnist/

I still don't know how to get working this: I'm trying to use my own database of images and labels, but the script asks me for:

parse_mnist_labels("train-labels.idx1-ubyte", &train_labels);
parse_mnist_images("train-images.idx3-ubyte", &train_images);
parse_mnist_labels("t10k-labels.idx1-ubyte", &test_labels);
parse_mnist_images("t10k-images.idx3-ubyte", &test_images);

and I don't know how to convert from my database to those kind of files.

Does anyone have a clue ?
Thank you very much.

Have constant values on predict

Hello!
I have such a problem: when I'm trying to predict class, I get the same value vector, no matter, what input data there is.
As input the net has 32x32 image. As output - two classes, or simplier - just yes/no.
So, I always get output answer smth like this: [0.8, -0.8].
After training I get a file which looks like good (many different numbers).
What could I forget?
Thank you.

Source code:

// NeuralNetwork::fNet has type network<mse, adagrad>

void NeuralNetwork::CreateNet()
{
    fNet << convolutional_layer<tan_h>(32, 32, 5, 1, 6) // C1, 1@32x32-in, 6@28x28-out
       << average_pooling_layer<tan_h>(28, 28, 6, 2) // S2, 6@28x28-in, 6@14x14-out
       << convolutional_layer<tan_h>(14, 14, 5, 6, 16) // C3, 6@14x14-in, 16@10x10-in
       << average_pooling_layer<tan_h>(10, 10, 16, 2) // S4, 16@10x10-in, 16@5x5-out
       << convolutional_layer<tan_h>(5, 5, 5, 16, 120) // C5, 16@5x5-in, 120@1x1-out
       << fully_connected_layer<tan_h>(120, 2); // F6, 120-in, 2-out
}

void NeuralNetwork::Train(string trainDataFilename, string neuralNetFilename)
{
    CreateNet(32);

    vector<label_t> Y;
    vector<vector<double>> X;

        // This function is OK 100%
    ReadTrainData(trainDataFilename, X, Y);

    // create callback
    auto on_enumerate_epoch = [&](){
        nEpoch++;

        SYSTEMTIME now;
        GetSystemTime(&now);
        cout << endl << "Finished epoch #" << nEpoch << " - " << now.wHour << ":" << now.wMinute << endl;
    };

    auto on_enumerate_minibatch = [&](){
        cout << ".";
    };

    int minibatch_size = 10;
    int num_epochs = 30;
    fNet.optimizer().alpha *= std::sqrt(minibatch_size);
    fNet.train(X, Y, minibatch_size, num_epochs, on_enumerate_minibatch, on_enumerate_epoch);

    ofstream os(neuralNetFilename); 
    fNet.save(os);
    os.close();

    FilePath = neuralNetFilename;
}

void NeuralNetwork::LoadFromFile(string netFilename)    
{
    CreateNet(32);
    if (!netFilename.empty())
    {
        ifstream ifs(netFilename);
        if (ifs.is_open())
        {
            fNet.load(ifs);
            ifs.close();
        }
    }
}

label_t NeuralNetwork::Predict(vector<double>& x)
{
    if (IsReady())
    {
        auto y = fNet.predict(x);

        double maxval = -1000;
        label_t maxi = -1;
        for (int i = 0; i < y.size(); ++i)
        {
            if (y[i] > maxval)
            {
                maxval = y[i];
                maxi = i;
            }
        }
        return maxi;
    } else {
        return -1000000;
    }
}

Unable to call the predict() method

I get the following error when I try to predict the label for an image. I convert the image to a vector<vec_t> using the convert_image() method. There is a type mismatch problem that I keep getting. I've attached the full error message below. Many thanks for your help!

error: no matching function for call to ‘tiny_cnn::network<tiny_cnn::mse, tiny_cnn::RMSprop>::predict(std::vectorstd::vector&)’
nn.predict(attempt);
^
/home/ubuntu/Project/temp.cpp:100:20: note: candidate is:
In file included from /home/ubuntu/Project/../tiny-cnn/tiny_cnn/tiny_cnn.h:31:0,
from /home/ubuntu/Project/temp.cpp:9:
/home/ubuntu/Project/../tiny-cnn/tiny_cnn/network.h:109:18: note: tiny_cnn::vec_t tiny_cnn::network<LossFunction, Optimizer>::predict(const vec_t&) [with LossFunction = tiny_cnn::mse; Optimizer = tiny_cnn::RMSprop; tiny_cnn::vec_t = std::vector]
vec_t predict(const vec_t& in) { return fprop(in); }
^
/home/ubuntu/Project/../tiny-cnn/tiny_cnn/network.h:109:18: note: no known conversion for argument 1 from ‘std::vectorstd::vector’ to ‘const vec_t& {aka const std::vector&}’

Is there a bug in TEST(multi_layer, gradient_check)

I run the test project,failed to pass TEST(multi_layer, gradient_check) use the following code.

When the size of the hidden layer of the multi_layer network is small it works,but fails when the size greater than 100.


TEST(multi_layer, gradient_check) { // sigmoid - cross-entropy
    typedef cross_entropy loss_func;
    typedef sigmoid activation;
    typedef network network;

    network nn;
/*
nn << fully_connected_layer(3, 100)
<< fully_connected_layer(100, 2);
    */
//above can pass the gradient check,but the fllowing can't
    nn << fully_connected_layer(3, 101)
        << fully_connected_layer(101, 2);

    vec_t a(3, 0.0);
    label_t t = 1;

    uniform_rand(a.begin(), a.end(), 0, 1);
    nn.init_weight();
    EXPECT_TRUE(nn.gradient_check(&a, &t, 1, 1e-4, GRAD_CHECK_ALL));
}


may consume too much memory when creating big network

Hi, I'm trying to create a NN like :

    nn << convolutional_layer<tan_h>(320, 320, 33, 1, 20)
        << average_pooling_layer<tan_h>(288, 288, 20, 2)
        << convolutional_layer<tan_h>(144, 144, 15, 20, 30)
        << average_pooling_layer<tan_h>(130, 130, 30, 2)
        << convolutional_layer<tan_h>(65, 65, 6, 30, 20)
        << max_pooling_layer<tan_h>(60, 60, 20, 3)
        << fully_connected_layer<tan_h>(8000, 558);

but I got "vector subscript out of range" error, it seems like that index3d.size() doesn't return right value when in_ is defined as index3d< unsigned short > type, so I change code:

"typedef unsigned short layer_size_t;" => "typedef unsigned int layer_size_t;"

but this time, the program is eating memory until whole memory was occupied. (i'm using a 8G RAM)
I think the implement of partial_connected_layer.h is straightforward and intelligible but it may cosume too much memory when creating a big network because of "io/wi/wo_connections".

Querying locations of objects...

Is it possible to locate specific feature in an image and get their locations from tiny-cnn?

If I train a NN to detect say sail boats... and then I get a positive result... is their a way to determine what part of the image contributed to the positive result?

How should I create the dataset files?

I have images of faces. Images are of 64x64 dimensions. I used MNISTEN to build the file training and test both. Now when I pass this file to the convnet I get an exception:

throw nn_error("input dimension mismatch");

nn << convolutional_layer<tan_h>(64, 64, 7, 1, 6)
<< average_pooling_layer<tan_h>(58, 58, 6, 2)
<< convolutional_layer<tan_h>(29, 29, 8, 6, 16, connection_table(connection, 6, 16))
<< average_pooling_layer<tan_h>(22, 22, 16, 2) //S4
<< convolutional_layer<tan_h>(11, 11, 11, 16, 180) //C5
<< fully_connected_layer<tan_h>(180, 15);//F6

I am referring to this paper.

http://dap.vsb.cz/wsc17conf/Media/Default/Page/online_wsc17_submission_59.pdf

Kindly suggest and point out where I could be wrong.

My dataset is perfect, all the images are of 64x64.

Also I tried using 32x32 images on Le net version here .

Please provide a solution. What could be possibly wrong.

Adding parameter beyond pixel data

In the LeNet example:

https://github.com/nyanp/tiny-cnn/wiki/MNIST-Example

Is there any way to add a parameter beyond just the raw pixel data? For example, I would like to add in the (x, y) coordinates of the given image within a larger image. Presumably this would be added on the last layer, the fully connected layer, as additional 2 inputs. I have no idea how to do this, or if it is possible.

How to use weigths

We would like to use network's weights, obtained by using tiny-cnn in our test program. How do you write trained weights in file? What sequence?
How it will look for an example net:
net << convolutional_layer<tan_h>(32, 32, 5, 1, 6) // 32x32in, conv5x5, 1-6 f-maps
<< average_pooling_layer<tan_h>(28, 28, 6, 2) // 28x28in, 6 f-maps, pool2x2
<< fully_connected_layer<tan_h>(14 * 14 * 6, 120)
<< fully_connected_layer<tan_h>(120, 10);

Thank you!

Check for -1.#IND value when saving weights

Please add some basic check for invalid floating point values when saving network weights.
My trained CNN has some -1.#IND weight values, after saving them into a text file, they can't be loaded by network::load function.
Here is my solution to fix this problem:
in class layer_base::

virtual void save(std::ostream& os) const {
    for (auto w : W_) os << (w == w ? w : 0.0) << " ";
    for (auto b : b_) os << (b == b ? b : 0.0) << " ";
}

Data Normalization

Hello:
thanks for sharing your code,I use opencv to convert the face images to vect_t, which value is from 0 to 255, I want to know wheather I need to normalize the data from 0 to 1? thanks very much.

update_weight function

Dear nyanp,

I'm sorry that I created bug report, unfortunately I did not find any other way to communicate with you. (I found your website and page in social network, but there are no email, Skype etc.)
If it is not hard for you, could you explain, please how work update_weight function?
I can not understand how you calculate dW_, Whessian_ and so on.

Sorry for disturb you!

Best regards,
Paul

How to compile in Ubuntu 14.04?

I installed tbb and boost by apt-get but at the configuration stage it does not alert anything about these libraries. It outputs same as I ./waf configure alone. Is this the ./waf configure output with correct settings of tbb and boost libs?

Setting top to : /home/bla/tiny-cnn-master
Setting out to : /home/bla/tiny-cnn-master/build
Checking for program g++,c++ : g++
Checking for program ar : /usr/bin/ar
'configure' finished successfully (0.025s)

How CIFAR 10 dataset can be used in tiny-cnn like the MNIST dataset?

I have the binary version of the dataset.
The binary version contains the files data_batch_1.bin, data_batch_2.bin, ..., data_batch_5.bin, as well as test_batch.bin. Each of these files is formatted as follows:
<1 x label><3072 x pixel>
...
<1 x label><3072 x pixel>
In other words, the first byte is the label of the first image, which is a number in the range 0-9. The next 3072 bytes are the values of the pixels of the image. The first 1024 bytes are the red channel values, the next 1024 the green, and the final 1024 the blue. The values are stored in row-major order, so the first 32 bytes are the red channel values of the first row of the image.

Each file contains 10000 such 3073-byte "rows" of images, although there is nothing delimiting the rows. Therefore each file should be exactly 30730000 bytes long.

There is another file, called batches.meta.txt. This is an ASCII file that maps numeric labels in the range 0-9 to meaningful class names. It is merely a list of the 10 class names, one per row. The class name on row i corresponds to numeric label i.

compiling and running /example/main.cpp - getting 9% accuracy!

Hi,
Thanks for sharing this tiny-CNN.

I compiled /example/main.cpp with TBB and SSE options enabled. I did not make any changes to "main.cpp". When the run the compiled executable, I get an accuracy of 9.58%!. Looking at the confusion matrix it appears that the network classifies every test sample as number "6"!

I am not sure if I am doing something wrong or not.

reduce template parameter (CRTP)

    typedef network<mse, gradient_descent_levenberg_marquardt> CNN;
    CNN nn;
    convolutional_layer<CNN, identity> layer(5, 5, 3, 1, 1);
    nn.add(&layer);

should be

    network<mse, gradient_descent_levenberg_marquardt> nn;
    convolutional_layer<identity> layer(5, 5, 3, 1, 1);
    nn.add(&layer);

run tiny-cnn on my mac terminal

can i run tiny-can on my mac terminal?
i have tried and i have installed C++ boost library. However, when i tried ''./waf build'', there is an error:


wlib-d-116:tiny-cnn-master gary$ ./waf build
Waf: Entering directory `/Users/gary/Desktop/tiny-cnn-master/build'
[2/2] Linking build/example/main
ld: library not found for -lboost_timer-mt
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Waf: Leaving directory `/Users/gary/Desktop/tiny-cnn-master/build'
Build failed
-> task in 'main' failed (exit status 1):
{task 4340392080: cxxprogram main.cpp.1.o -> main}

['/usr/bin/clang++', 'example/main.cpp.1.o', '-o', '/Users/gary/Desktop/tiny-cnn-master/build/example/main', '-L../', '-lstdc++', '-lboost_timer-mt', '-ltbb']

Is this mean that i didn't successfully install boost?
how can i do??

tiny-cnn community...

First let me say that I love tiny-cnn!

But, I have about a million questions. I think we need a board, or maybe a google group we can post questions to... Also, a wiki for sharing techniques might be nice.

Importing weights

Hi,
we are trying to re-implement a cnn previously trained using keras. We are wondering in which order we should write the weights on a file (a txt file?) in order to be able to import them in a tiny-cnn net.
Unfortunately we couldn't find any documentation about this topic.

CMakeLists.txt out of sync

Looks like you need to update the CMakeLists.txt to accomadate the re-factored library directory structure. i.e. tiny_cnn/convolutional_layer.h to tiny_cnn/io/convolutional_layer.h , etc.

./waf build failed

I installed tiny-cnn using the command:./waf configure --TBB --TBB_ROOT=/usr/include/tbb --BOOST_ROOT=/usr/include/boost,
which ouputed:
Setting top to : /home/chensq/tiny-cnn-master
Setting out to : /home/chensq/tiny-cnn-master/build
Checking for 'g++' (C++ compiler) : /usr/bin/g++
'configure' finished successfully (0.012s)
Then, I inputed the command: ./waf build, which outputed:
Waf: Entering directory `/home/chensq/tiny-cnn-master/build'
[1/2] Compiling example/main.cpp
../example/main.cpp:31:22: fatal error: tiny_cnn.h: No such file or directory
#include "tiny_cnn.h"
^
compilation terminated.

Waf: Leaving directory `/home/chensq/tiny-cnn-master/build'
Build failed
-> task in 'main' failed (exit status 1):
{task 139992378742224: cxx main.cpp -> main.cpp.1.o}
['/usr/bin/g++', '-std=c++0x', '-Wall', '-s', '-Ofast', '-I/home/chensq/tiny-cnn-master/build/example', '-I/home/chensq/tiny-cnn-master/example', '-I/home/chensq/tiny-cnn-master/build/include', '-I/home/chensq/tiny-cnn-master/include', '-I/usr/include/boost', '-I/usr/include/tbb', '-DCNN_USE_TBB=""', '../example/main.cpp', '-c', '-o', '/home/chensq/tiny-cnn-master/build/example/main.cpp.1.o']

So I copyed all .h files which are in the tiny_cnn folder into example folder, then I ./waf build again, which ouputed:
Waf: Entering directory `/home/chensq/tiny-cnn-master/build'
[2/2] Linking build/example/main
/usr/bin/ld: cannot find -lboost_timer-mt
collect2: error: ld returned 1 exit status

Waf: Leaving directory `/home/chensq/tiny-cnn-master/build'
Build failed
-> task in 'main' failed (exit status 1):
{task 139977480258960: cxxprogram main.cpp.1.o -> main}
['/usr/bin/g++', 'example/main.cpp.1.o', '-o', '/home/chensq/tiny-cnn-master/build/example/main', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L../', '-lstdc++', '-lboost_timer-mt', '-ltbb']
My gcc version is 4.8.2 in the Ubuntu 14.04.
Could you help me? Thank you in advance!

Dropout

Is dropout support fully implemented?

I notice that convolutional_layer<> takes an optional Filter template argument and then does nothing with it. Also, I am running into trouble using "fully_connected_dropout_layer" on the upper levels of my network. A convolutional_layer( 64, 64, 7, 3, 16 ) requires a fully_connected_dropout_layer with 53824 inputs and outputs... and I run out of memory. Am I using this wrong?

memory leak in network class

Here is the code in network.h to add a temporarily-constructed layer to the network:

template <typename L, typename O, typename Layer>
network<L, O>& operator << (network<L, O>& n, const Layer&& l) {
    n.add(new Layer(l));
    return n;
}

The "new"ed layer is not deleted when the network object is destructed, causing memory leak.
Please consider storing shared smart pointers of layers instead of raw pointers inside a network object.

Can I use tiny-cnn for classifying human and car?

Dear nyanp:
Tiny-cnn is a CNN of smart and excellent which can be used in Windows. I've been studying to it for several days, and I got some problems recently.

I try to use tiny-cnn for classifying human and car. I label images of human to 0 and car to 1. These images are convert to gray-scale and resize to 32x32. Default setting has been used but the output dimension of the last fully connected layer has been changed to 2.

All images are took from surveillant video, but images of human are from one camera and cars are from another. The subject occupies the main space in every image.

Now something strange happens. All the outputs of testing set are 1, i.e. car. I have no idea wether problem comes from data set or settings of tiny-cnn.

If more detail is needed, I'll reply as far as possible. Thanks very much!

Best regards,
younglittlefat

read the trained model and predict error

I have a trained model and saved it. But a mistake happened when I predict a test sample's class attribute. Of course, I have load the model before the prediction correctly. The debugging information gave the tips "the input layer's size is empty", I donnot understand the information, because the entire proccess is right!

How use TBB to complie the project?

Hi,I'm very grateful to your work.It's so nice.I have a work which need the cnn.because of the big data,I want to try the TBB,but I know little about TBB and the knowledge is so absent.I want to know the detail steps to complie and run the project. I would be very appreciative to you if you can give me some tips.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.