m2dsupsdlclass / lectures-labs Goto Github PK

Slides and Jupyter notebooks for the Deep Learning lectures at Master Year 2 Data Science from Institut Polytechnique de Paris

License: MIT License

HTML 0.72% CSS 0.49% Jupyter Notebook 98.15% Python 0.64% SCSS 0.01%

deep-learning neural-network jupyter-notebook slide python lecture

lectures-labs's Introduction

Deep Learning course: lecture slides and lab notebooks

This course is being taught at as part of Master Year 2 Data Science IP-Paris

The course covers the basics of Deep Learning, with a focus on applications.

Lecture slides

Note: press "P" to display the presenter's notes that include some comments and additional references.

Lab and Home Assignment Notebooks

The Jupyter notebooks for the labs can be found in the labs folder of the github repository:

git clone https://github.com/m2dsupsdlclass/lectures-labs

These notebooks only work with keras and tensorflow Please follow the installation_instructions.md to get started.

Direct links to the rendered notebooks including solutions (to be updated in rendered mode):

Lab 1: Intro to Deep Learning

Lab 2: Neural Networks and Backpropagation

Lab 3: Embeddings and Recommender Systems

Lab 4: Convolutional Neural Networks for Image Classification

Lab 5: Deep Learning for Object Dection and Image Segmentation

Lab 6: Text Classification, Word Embeddings and Language Models

Lab 7: Sequence to Sequence for Machine Translation

Translation of Numeric Phrases with Seq2Seq

Lab 8: Intro to PyTorch

Lab 9: Siamese Networks and Triplet loss

Lab 10: Variational Auto Encoder

VAE on Fashion MNIST

Acknowledgments

This lecture is built and maintained by Olivier Grisel and Charles Ollion

Charles Ollion, head of research at Heuritech - Olivier Grisel, software engineer at Inria

We thank the Orange-Keyrus-Thalès chair for supporting this class.

License

All the code in this repository is made available under the MIT license unless otherwise noted.

The slides are published under the terms of the CC-By 4.0 license.

lectures-labs's People

Contributors

Stargazers

Watchers

Forkers

tizot mrastgoo jinyu0310 azotlikid robertocastellini87 farahzack challier salehmoe snakeroot91 ravs1 orapradeep aimran maxim-popkov aaskat davidrhodus w4zir jiandai balinus dondanidang xiaozan-pku sunjieee allensmile yiliu001 houtianqi falconzyx vybhavk rohit22 anuragreddygv323 sposman2 ccaleanu ajwije azoryk andandandand thanhlct dahernan novelmartis tdhopper xypan1232 namanj wanjinchang abyoussef hoardboard sandy4321 vgoklani benjamesbabala hxl1990 junedylan sebastienboisard hello-kukoo ml-lab ml-ai-nlp-ir richa10 amiltonwong sethu424 irankunda d0z1ngshark little1tow xc35 tien-le-grenoble smgaya miguelperalvo paul-darius pierrearb robinroyer blanchonmarc cttcpp krsnaa bigsnarfdude vyraun jinkos vichuda yanghaha11514 jeremygu hejunbok gandalfvn wuzhongdehua jellyang virajago ekalyoncu nazben micseb fengwangjiang longfly2 nlpformyself nbara yatiraj1 melaniefns decebel m7catsue hailingc sayantanmukherjee6 bird1235456 mageed xiexiaoshinick leezqcst drauh merico34 lepy adri-mash resurgo-genetics

lectures-labs's Issues

Clarification on embedding input dimension for Recommender system

Dear Olivier,

Thank you for the lab tutorial on recommender system, it was really helpful.

While going through and study your lab tutorial on Explicit Feedback Neural Recommender System : https://github.com/m2dsupsdlclass/lectures-labs/blob/master/labs/03_neural_recsys/Explicit_Feedback_Neural_Recommender_System_rendered.ipynb

I would like to check with you whether there is a mistake on the code below :

self.user_embedding = Embedding(output_dim=embedding_size,
input_dim=max_user_id + 1,
input_length=1,
name='user_embedding')
self.item_embedding = Embedding(output_dim=embedding_size,
input_dim=max_item_id + 1,
input_length=1,
name='item_embedding')

Should the input_dim be max_user_id (instead of max_user_id+1) since the dataset is only start from 1 to 943 , ie there are total of 943 user in the dataset hence there is no need to + 1 ?

min_user_id = all_ratings['user_id'].min()
min_user_id
Out[7]:
1

max_user_id = all_ratings['user_id'].max()
max_user_id
Out[8]:
943

The same applies to the item_embedding as well.

Look forward to your reply.

Thanks

may you share links to video of classes

very good material
thanks a lot
only it would be very kind of you to share links to video of classes
thanks

Error in solution grl_training: ValueError: Found two metrics with the same name: digits_classifier_accuracy

If running the lab for the domain adaptation and using the grl_training solution the model gets trained without errors but the following error occurs if model.evaluate(...) is called after the training:
ValueError: Found two metrics with the same name: digits_classifier_accuracy

Does anyone can help to find out why?

SoftmaxMap/Softmax4d seems not working for the newest version of Keras?

Hi,

I'm using the 'SoftmaxMap' of here (I've modified it slightly in accordance with the Keras Documentation):

from keras.engine import Layer
import keras.backend as K

class SoftmaxMap(Layer):
    # Init function
    def __init__(self, axis=-1, **kwargs):
        self.axis = axis
        super(SoftmaxMap, self).__init__(**kwargs)
    def build(self,input_shape):
        pass
    def call(self, x, mask=None):
        e = K.exp(x - K.max(x, axis=self.axis, keepdims=True))
        s = K.sum(e, axis=self.axis, keepdims=True)
        return e / s
    def compute_output_shape(self, input_shape):
        return input_shape

I'm doing this because I'd like to run a fully convolutional network with Keras (see the following for the network structure)

model = Sequential()

#conv1
model.add(Conv2D(filters=96, kernel_size=(11, 11),
                 strides=(4,4),
                 padding='valid',
                 input_shape=(None,None,3)
                )
         )
model.add(Activation('relu'))
#pooling1
model.add(MaxPooling2D(pool_size=(3, 3),
                       strides=(2,2),
                       padding='valid'
                      )
         )
#conv2
model.add(Conv2D(filters=256, kernel_size=(5, 5),
                 strides=(1,1),
                 padding='same'
                )
         )
model.add(Activation('relu'))
#pooling2
model.add(MaxPooling2D(pool_size=(3, 3),
                       strides=(2,2),
                       padding='valid'
                      )
         )
#conv3
model.add(Conv2D(filters=384, kernel_size=(3, 3),
                 strides=(1,1),
                 padding='same'
                )
         )
model.add(Activation('relu'))
#conv4
model.add(Conv2D(filters=384, kernel_size=(3, 3),
                 strides=(1,1),
                 padding='same'
                )
         )
model.add(Activation('relu'))
#conv5
model.add(Conv2D(filters=256, kernel_size=(3, 3),
                 strides=(1,1),
                 padding='same'
                )
         )
model.add(Activation('relu'))
#pooling3
model.add(MaxPooling2D(pool_size=(3, 3),
                       strides=(2,2),
                       padding='valid'
                      )
         )
#conv6
model.add(Conv2D(filters=4096, kernel_size=(6, 6),
                 strides=(1,1),
                 padding='valid'
                )
         )
model.add(Activation('relu'))
#conv7
model.add(Conv2D(filters=4096, kernel_size=(1, 1),
                 strides=(1,1),
                 padding='valid'
                )
         )
model.add(Activation('relu'))
#conv8
model.add(Conv2D(filters=2, kernel_size=(1, 1),
                 strides=(1,1),
                 padding='valid'
                )
         )
#model.add(Flatten())
#model.add(GlobalAveragePooling2D())
#model.add(Activation('softmax'))
    
model.add(SoftmaxMap(axis=-1))

model.compile(loss='categorical_crossentropy',
              optimizer=SGD(lr=0.005,momentum=0.3),
              metrics=['accuracy'])

However, I got an error while fitting the model:

ValueError: Error when checking target: expected softmax_map_1 to have 4 dimensions, but got array with shape (128, 2)

which is quite weird. I'm using Tensorflow 1.2.1. Any suggestions?

Predict_classes

'Sequential' object has no attribute 'predict_classes'

This function were removed in TensorFlow version 2.6.