morvanzhou / pytorch-tutorial Goto Github PK

https://github.com/menorashid/animal_human_kp

你好莫烦，首先谢谢你带给我们的课程，感谢你提供给我的帮助，我也都关注你了

目前我进行的研究需要用到以上的github库，那个库的数据标注文件是npy格式的，想请教一下莫烦，这种npy格式文件是如何生成的，如果想要制作自己的数据集，应该怎么做？

谢谢

Really appreciate for your effort in making wonderful tutorials on deep learning. Your work is outstanding, in its clarity and wide coverage of examples and libraries. I have explored a lot of deep learning tutorials and I like yours a lot.

I sincerely hope your work will keep expanding and growing. I would like to know your teaching plan for the future, because I intend to spend lots of time studying your tutorials carefully.

As you know, examples are great tools to learn, simple examples are necessary to get people started, and wide range and in-depth examples are also important to keep people motivated and going forward. I would like to see your tutorials and videos moving on to cover more examples while keeping the dataset small enough for Mac cpu to handle within reasonable time.

I think you are already a great hit in deep learning tutorials in Chinese language, and keep going like this personally I believe you will be more famous than Siraj (who is currently one of the hottest deep learning evangelists)

All the best

122行的代码：accuracy = sum(pred_y == test_y) / float(test_y.size(0)) 在0.4.0的新版本下运行报错：“RuntimeError: value cannot be converted to type uint8_t without overflow: 2000.000000”
并且sum(pred_y == test_y) 求出来的值也不是pred_y和test_y相等的值（会小很多）
我试了下把sum(pred_y == test_y) 改成 torch.sum(pred_y == test_y).numpy()，可以求出正确结果，但是我不知道是为什么前面的代码不能用了，莫凡大神能解答一下吗？谢谢！

why do u have

Why do u have 'setattr(self, 'fc%i' % i, fc)'

in ur code? is it really necessary? the standard cifar10 tutorial doesn't have it...

https://github.com/MorvanZhou/PyTorch-Tutorial/blob/master/tutorial-contents/504_batch_normalization.py

hi,@莫凡，我使用mseloss来计算两副图像，尺寸为(Batch,2,101,200)；然后，我无论怎么更改mseloss中的三个参数为true或者false，loss=6*10^8不变。

很不好意思，提出的问题不是您文档中的，但是依然渴求解答。

I tried some RNN regression learning based on the code in the "PyTorch-Tutorial/tutorial-contents/403_RNN_regressor.py" file, which did not work for me at all.

According to an accepted answer on stack-overflow (https://stackoverflow.com/questions/52857213/recurrent-network-rnn-wont-learn-a-very-simple-function-plots-shown-in-the-q?noredirect=1#comment92916825_52857213), it turns out that the line

h_state = h_state.data

of the tutorial file does not do what it is supposed to be doing. It does not actually break the connection to the previous iteration as specified in the commment. Rather the line

h_state = None

should do the trick (according to the accepted answer).

我运行这个源码。python2.7
出现错误：Traceback (most recent call last):
File "./LR_test.py", line 51, in
plt.scatter(x.data.numpy(), y.data.numpy())
RuntimeError: PyTorch was compiled without numpy support
但是我安装的时候运行了conda install numpy pyyaml mkl setuptools cmake cffi这句话，这是什么原因啊

I try to run 405_dqn_rl,while encountering this error. How to solve this question?
RuntimeError: tried to construct a tensor from a nested int sequence, but found an item of type numpy.int32 at index (0, 0)

Thanks to the effort of Morvanzhou, I got a wonderful start of deep learning,but for this one, code403, I got a little confused that how to change these codes of RNN into the kind of LSTM.I wish you can show it to me,thanks a lot.

I try to install pytorch and after days of trying Im here with a big, big problem. I read a lot of articles of "how to install pytorch" I try to install with pip install but dont work for me and after I install it with Anaconda, but in anaconda is pytorch install, when I type: conda list, he is there like this form: pytorch 1.0.1 py3.7_cuda100_cudnn7_1 pytorch, I have python 3.7, when I run a code with import torch this show me a message like this:

And when i try to import torch in python 3.7:

Pip install error:

How to pass this errors? Please Help, thx.

Hi, I run this code,but has a error,the outpout is:

Please install sklearn for layer visualization
Traceback (most recent call last):
File "/home/hang/PycharmProjects/pytorch/test.py", line 209, in
output = cnn(b_x)[0] # cnn output
File "/home/hang/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in call
result = self.forward(*input, **kwargs)
File "/home/hang/PycharmProjects/pytorch/test.py", line 180, in forward
x = self.conv1(x)
File "/home/hang/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in call
result = self.forward(*input, **kwargs)
File "/home/hang/.local/lib/python2.7/site-packages/torch/nn/modules/container.py", line 67, in forward
input = module(input)
File "/home/hang/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in call
result = self.forward(*input, **kwargs)
File "/home/hang/.local/lib/python2.7/site-packages/torch/nn/modules/conv.py", line 282, in forward
self.padding, self.dilation, self.groups)
File "/home/hang/.local/lib/python2.7/site-packages/torch/nn/functional.py", line 90, in conv2d
return f(input, weight, bias)
TypeError: argument 0 is not a Variable

Process finished with exit code 1

还有一个问题，为什么代码与视频中的不一样？
是因为Variable的问题吗
谢谢

Thanks for sharing this tutorial code.
One question is that your 302_classification.py file does not contain a 'F.softmax()' function, which is inconsistent with the situation in your Youtube video.

D_loss.backward(retain_variables=True) # retain_variables for reusing computational graph

got an error:

TypeError Traceback (most recent call last)
in ()
11
12 opt_D.zero_grad()
---> 13 D_loss.backward(retain_variables=True) # retain_variables for reusing computational graph
14 opt_D.step()
15

TypeError: backward() got an unexpected keyword argument 'retain_variables'

In pytorch-tutorial,I try to run 302_classification.py,and something go run:
raise ValueError("x and y must be the same size")
ValueError: x and y must be the same size
I don't get it.

我一开始以为两者区别是灰度图只有一个通道只需要conv2d 如果是彩色RGB需要conv3d
但是仔细看了下conv2d里面也是包含了in_channels的 也就是conv2d是可以对彩色RGB进行卷积的。
不知道我这样理解是否正确。
那么conv3d又是什么场合使用？

问题如下：
（除此之外代码还有好多其他的问题。。我2.7和3.6都试了，不管用。。。谢谢老师。。。）
TypeError Traceback (most recent call last)
in
97 for step, (b_x, b_y) in enumerate(train_loader): # gives batch data, normalize x when iterate train_loader
98
---> 99 output = cnn(b_x)[0] # cnn output
100 loss = loss_func(output, b_y) # cross entropy loss
101 optimizer.zero_grad() # clear gradients for this training step

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
355 result = self._slow_forward(*input, **kwargs)
356 else:
--> 357 result = self.forward(*input, **kwargs)
358 for hook in self._forward_hooks.values():
359 hook_result = hook(self, input, result)

in forward(self, x)
68
69 def forward(self, x):
---> 70 x = self.conv1(x)
71 x = self.conv2(x)
72 x = x.view(x.size(0), -1) # flatten the output of conv2 to (batch_size, 32 * 7 * 7)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
355 result = self._slow_forward(*input, **kwargs)
356 else:
--> 357 result = self.forward(*input, **kwargs)
358 for hook in self._forward_hooks.values():
359 hook_result = hook(self, input, result)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torch/nn/modules/container.py in forward(self, input)
65 def forward(self, input):
66 for module in self._modules.values():
---> 67 input = module(input)
68 return input
69

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
355 result = self._slow_forward(*input, **kwargs)
356 else:
--> 357 result = self.forward(*input, **kwargs)
358 for hook in self._forward_hooks.values():
359 hook_result = hook(self, input, result)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torch/nn/modules/conv.py in forward(self, input)
280 def forward(self, input):
281 return F.conv2d(input, self.weight, self.bias, self.stride,
--> 282 self.padding, self.dilation, self.groups)
283
284

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torch/nn/functional.py in conv2d(input, weight, bias, stride, padding, dilation, groups)
88 _pair(0), groups, torch.backends.cudnn.benchmark,
89 torch.backends.cudnn.deterministic, torch.backends.cudnn.enabled)
---> 90 return f(input, weight, bias)
91
92

TypeError: argument 0 is not a Variable

新版本的torch
Variable（）函数将返回tensor，并且无需requires_grad了
可以更新下代码啦

如题。请问在训练过程中，batch_size一般为1吗？因为不同的样本似乎有不同的动作选择次数，一次性多个batch训练是不可行的，所以batch_size一般都是1，对吗？

讲的确实很好但是我们这些新手更希望和实际结合起来，这样才能刚快的学习各种概念

比如在cnn那一节 ，mnist的训练结果如果保存了下来想做自己写个数字检验下结果这个数据预处理的步骤，一起结合起来就好了

现在我就遇到了这样的问题，使用mnist的结果想要预测自己任意找到的一张图片但是数据预处理的结果和torch希望得到的结果不一样，现在很苦恼

nn/modules/conv.py", line 301, in forward self.padding, self.dilation, self.groups) RuntimeError: Expected 4-dimensional input for 4-dimensional weight [16, 1, 5, 5], but got input of size [26, 28] instead

可以看到上面，需要一个四维的数据，如果可以再串起来讲下这个就好了

另一个问题 如何解决这个四维的问题呢？

莫烦大神，您知道怎么处理dicom格式的CT图像吗？怎么制作把CT图像制作成自己的数据集来用来分类呢？关于肺结节的分类您有没有相关的研究？

https://github.com/MorvanZhou/PyTorch-Tutorial/blob/9318a69f081c5a4074a442fc2a297e6da187a5e3/tutorial-contents/401_CNN.py#L65

The comment should be "# input shape (16, 14, 14)"

Add a line statement below the end of Net class：
if name == 'main':

Then indent the code below。
Search the code l.append(loss_func(pred, test_y).data[0]) replace .data[0] with .item() to fix the waring
E:/code/python/learnPytorch/test1.py:121: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
l.append(loss_func(pred, test_y).data[0])

RuntimeError: input and target shapes do not match: input [10 x 1], target [1 x 10 x 1]

I think
y = Variable(torch.from_numpy(y_np[np.newaxis, :, np.newaxis]))
should be
y = Variable(torch.from_numpy(y_np[:, np.newaxis]))

could you also update your code to be compatible to version 0.4.0. I mean removing Variable

Thank you

你好，
文件tutorial-contents/403_RNN_regressor.py 中的这段代码

r_out = r_out.view(-1, 32)
outs = self.out(r_out)
return outs, h_state

outs的shape是 10x1, 与(batch, time_step, input_size) 不匹配， 导致RuntimeError: input and target shapes do not match: input [10 x 1], target [1 x 10 x 1]
是否可以这样直接添加一个维度，匹配(batch, time_step, input_size)

r_out = r_out.view(-1, 32)  
outs = self.out(r_out) 
return torch.unsqueeze(outs,0), h_state

201_torch_numpy.ipynb does not load in the GitHub interface, while 202_variable.ipynb opens without any problem

After I upgrade to pytorch 0.2

The example code in 405_DQN_Reinforcement_learning.py is broken.
This is because torch.max() change it's return.

So the code need to change to run in pytorch 0.2:

LN:62

action = torch.max(actions_value, 1)[1].data.numpy()[0, 0]

# change to
action = torch.max(actions_value, 1)[1].data.numpy()[0]

LN:90

 q_target = b_r + GAMMA * q_next.max(1)[0]

# change to 
 q_target = b_r + GAMMA * q_next.max(1)[0].view(32, 1)

Thanks for you tutorial, it help me a lot!!

In the code use rnn to find cos(x) by sin(x), the time_step = 1 and the batch_size = 1?

In the 10th cell, l_his.append(loss.data[0]) should be l_his.append(loss.item()) to suppress warning.

/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:14: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number

why do not you use F.softmax in follow code?

code: prediction = torch.max(out, 1)[1]

fixed

Besides bugs mentioned in #55,
print('Epoch: ', epoch, '| train loss: %.4f' % loss.data[0], '| test accuracy: %.2f' % accuracy)
should be replaced with
print('Epoch: ', epoch, '| train loss: %.4f' % loss.item(), '| test accuracy: %.2f' % accuracy)
to suppress the following warning:

/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:41: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number

Dear author:

In 403, when I commented h_state = h_state.data, it gave me the following error:

RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

And later I found the error can actually be solved by loss.backward(retain_graph=True).

I wonder what exactly does the error mean. And how does h_state = h_state.data solve the problem?

Thanks in advance.

Hello,Morvan
when i run the code in 305_batch_train.py, it throw an error. Then I comment num_workers=2, it successes.
torch version: 0.4.0 OS: windows

tutorial-contents/306_optimizer.py

hello，莫凡！如果不把主要代码放在 if name == 'main': 下，在windows+py3的环境下会报错

代码里在网络训练之前没有看到权重初始化的语句，不初始化权重的话都为0了，怎么训练的鸭？
或者是代码里用哪部分对权重初始化了？
谢谢~

Traceback (most recent call last):
File "/home/fanlu/PycharmProjects/learnTheano/firstCNN.py", line 111, in
output = cnn(b_x)[0]
File "/home/fanlu/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/fanlu/PycharmProjects/learnTheano/firstCNN.py", line 78, in forward
x = self.conv2(x),
File "/home/fanlu/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/fanlu/.local/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/fanlu/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/fanlu/.local/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 301, in forward
self.padding, self.dilation, self.groups)
TypeError: conv2d(): argument 'input' (position 1) must be Tensor, not tuple

注释有误，应为 (64, 1, 28, 28)

In the 7th cell of 502_GPU.ipynb,
accuracy = sum(pred_y == test_y) / test_y.size(0) should be replaced with accuracy = torch.sum(pred_y == test_y).type(torch.FloatTensor) / test_y.size(0), or it will always be 0.

ValueError: max() arg is an empty sequence

Hi,

I am getting the following error when I run the 401_CNN.ipynb. Please help me to resolve that.

Thank you in advance.

NameError Traceback (most recent call last)
in ()
34 low_dim_embs = tsne.fit_transform(last_layer.data.numpy()[:plot_only, :])
35 labels = test_y.numpy()[:plot_only]
---> 36 plot_with_labels(low_dim_embs, labels)
37 plt.ioff()

in plot_with_labels(lowDWeights, labels)
8 for x, y, s in zip(X, Y, labels):
9 c = cm.rainbow(int(255 * s / 9)); plt.text(x, y, s, backgroundcolor=c, fontsize=9)
---> 10 plt.xlim(X.min(), X.max()); plt.ylim(Y.min(), Y.max()); plt.title('Visualize last layer'); plt.show(); plt.pause(0.01)
11
12 plt.ion()

~/pytorch_python3/lib/python3.5/site-packages/matplotlib/pyplot.py in pause(interval)
292 canvas.start_event_loop(interval)
293 else:
--> 294 time.sleep(interval)
295
296

NameError: name 'time' is not defined

504_batch_normalization.py cannot plot dynamic histogram, could you help to check the code?

自己的深度学习模型可以导入已经预训练好的参数，采用如下的代码可以导入原模型与预训练模型名字相同层的参数。

pretrained_model = models.xxx(pretrained=True)
my_model = net()
pretrained_dict = pretrained_model.state_dict()
model_dict = my_model.state_dict()
pretrained_dict = {k: v for k, v in pretrained_dict.items() if k in model_dict}
model_dict.update(pretrained_dict)
my_model.load_state_dict(model_dict)

但是如果我的模型和预训练好的模型结构不是完全相同，例如，原预训练模型是若干个conv_2d直接连接而成，而我的模型在每两个conv_2d中间会有一个nn.BatchNorm层，这样导致想要导入参数的层和我模型中的层有错开的情况，再使用上述代码进行导入参数时，由于名字没有办法匹配上，所以就没法按照原先想的吧conv_2d的参数互相导入，请问我说的这种情况有办法解决吗？

当代码运行到89行时，程序报错slice indices must be integers or None or have an index method。但是如果将b_x转换为int类型会导致RuntimeError: Expected object of type torch.FloatTensor but found type torch.IntTensor for argument #4 'mat1'

您好，
关于代码110行，
我想问下为什么需要加个[0]，直接cnn(b_x)不行?

example

没法显示训练后的图像