gmalivenko / pytorch2keras Goto Github PK

View Code? Open in Web Editor NEW

849.0 14.0 145.0 243 KB

PyTorch to Keras model convertor

Home Page: https://pytorch2keras.readthedocs.io/en/latest/

License: MIT License

Python 100.00%

pytorch keras-model-convertor keras tensorflow onnx keras-models tensorflow-models resnet densenet machinelearning

pytorch2keras's Introduction

pytorch2keras

PyTorch to Keras model converter.

Installation

pip install pytorch2keras

Important notice

To use the converter properly, please, make changes in your ~/.keras/keras.json:

...
"backend": "tensorflow",
"image_data_format": "channels_first",
...

Tensorflow.js

For the proper conversion to a tensorflow.js format, please use the new flag names='short'.

Here is a short instruction how to get a tensorflow.js model:

First of all, you have to convert your model to Keras with this converter:

k_model = pytorch_to_keras(model, input_var, [(10, 32, 32,)], verbose=True, names='short')

Now you have Keras model. You can save it as h5 file and then convert it with tensorflowjs_converter but it doesn't work sometimes. As alternative, you may get Tensorflow Graph and save it as a frozen model:

# Function below copied from here:
# https://stackoverflow.com/questions/45466020/how-to-export-keras-h5-to-tensorflow-pb 
def freeze_session(session, keep_var_names=None, output_names=None, clear_devices=True):
    """
    Freezes the state of a session into a pruned computation graph.

    Creates a new computation graph where variable nodes are replaced by
    constants taking their current value in the session. The new graph will be
    pruned so subgraphs that are not necessary to compute the requested
    outputs are removed.
    @param session The TensorFlow session to be frozen.
    @param keep_var_names A list of variable names that should not be frozen,
                          or None to freeze all the variables in the graph.
    @param output_names Names of the relevant graph outputs.
    @param clear_devices Remove the device directives from the graph for better portability.
    @return The frozen graph definition.
    """
    from tensorflow.python.framework.graph_util import convert_variables_to_constants
    graph = session.graph
    with graph.as_default():
        freeze_var_names = \
            list(set(v.op.name for v in tf.global_variables()).difference(keep_var_names or []))
        output_names = output_names or []
        output_names += [v.op.name for v in tf.global_variables()]
        input_graph_def = graph.as_graph_def()
        if clear_devices:
            for node in input_graph_def.node:
                node.device = ""
        frozen_graph = convert_variables_to_constants(session, input_graph_def,
                                                      output_names, freeze_var_names)
        return frozen_graph


from keras import backend as K
import tensorflow as tf
frozen_graph = freeze_session(K.get_session(),
                              output_names=[out.op.name for out in k_model.outputs])

tf.train.write_graph(frozen_graph, ".", "my_model.pb", as_text=False)
print([i for i in k_model.outputs])

You will see the output layer name, so, now it's time to convert my_model.pb to tfjs model:

tensorflowjs_converter  \
    --input_format=tf_frozen_model \
    --output_node_names='TANHTObs/Tanh' \
    my_model.pb \
    model_tfjs

Thats all!

const MODEL_URL = `model_tfjs/tensorflowjs_model.pb`;
const WEIGHTS_URL = `model_tfjs/weights_manifest.json`;
const model = await tf.loadFrozenModel(MODEL_URL, WEIGHTS_URL);

How to use

It's the converter of PyTorch graph to a Keras (Tensorflow backend) model.

Firstly, we need to load (or create) a valid PyTorch model:

class TestConv2d(nn.Module):
    """
    Module for Conv2d testing
    """

    def __init__(self, inp=10, out=16, kernel_size=3):
        super(TestConv2d, self).__init__()
        self.conv2d = nn.Conv2d(inp, out, stride=1, kernel_size=kernel_size, bias=True)

    def forward(self, x):
        x = self.conv2d(x)
        return x

model = TestConv2d()

# load weights here
# model.load_state_dict(torch.load(path_to_weights.pth))

The next step - create a dummy variable with correct shape:

input_np = np.random.uniform(0, 1, (1, 10, 32, 32))
input_var = Variable(torch.FloatTensor(input_np))

We use the dummy-variable to trace the model (with jit.trace):

from pytorch2keras import pytorch_to_keras
# we should specify shape of the input tensor
k_model = pytorch_to_keras(model, input_var, [(10, 32, 32,)], verbose=True)

You can also set H and W dimensions to None to make your model shape-agnostic (e.g. fully convolutional netowrk):

from pytorch2keras.converter import pytorch_to_keras
# we should specify shape of the input tensor
k_model = pytorch_to_keras(model, input_var, [(10, None, None,)], verbose=True)

That's all! If all the modules have converted properly, the Keras model will be stored in the k_model variable.

API

Here is the only method pytorch_to_keras from pytorch2keras module.

def pytorch_to_keras(
    model, args, input_shapes=None,
    change_ordering=False, verbose=False, name_policy=None,
):

Options:

model - a PyTorch model (nn.Module) to convert;
args - a list of dummy variables with proper shapes;
input_shapes - (experimental) list with overrided shapes for inputs;
change_ordering - (experimental) boolean, if enabled, the converter will try to change BCHW to BHWC
verbose - boolean, detailed log of conversion
name_policy - (experimental) choice from [keep, short, random]. The selector set the target layer naming policy.

Supported layers

Activations:
- ReLU
- LeakyReLU
- SELU
- Sigmoid
- Softmax
- Tanh
Constants
Convolutions:
- Conv2d
- ConvTrsnpose2d
Element-wise:
- Add
- Mul
- Sub
- Div
Linear
Normalizations:
- BatchNorm2d
- InstanceNorm2d
Poolings:
- MaxPool2d
- AvgPool2d
- Global MaxPool2d (adaptive pooling to shape [1, 1])

Models converted with pytorch2keras

ResNet*
VGG*
PreResNet*
DenseNet*
AlexNet
Mobilenet v2

Usage

Look at the tests directory.

License

This software is covered by MIT License.

pytorch2keras's People

Contributors

Stargazers

Watchers

Forkers

mrlzla justasbr daijucug shubhampachori12110095 grseb9s josephch405 aliaksandrsiarohin codelinaro pkdogcom wayveai afcarl maksim-vatkin bassykuo vcchy cheesama sharib-vision tuboss nd1511 fitrialif fancyerii guanlongtianzi alexliyang freedom99 macos collawolley mapleandfire alzayats bealeson999 gonzalobailador andpol5 calclavia chipper1 eloizalczer aropan sun-yitao eltehupkes pranoot yunjieyin gaimjkp eycab gherao boozyguo gribml andrewssobral csaroff shuangliu1992 jimmy-ksu idearibosome chenxukun byfaith hisangke mattroz shiyuan0806 ps1td xiaowenjie210 sleepprogger jrch507 b2220333 michaelramamonjisoa sangkny anupgoenka grannas lnapoleao hjffily deftruth mortonjt zhly0 qureshizawar davideboschetto data-science-ai-open-source brucechou1983 sunpengfei1122 xll98 ericphilbaz alexw5702 joejiong mrmdzz agporto zhanguochang minygd felixzhang7 momingt vivekvinushanth wenbank mhrah7495 anindgupta junmugit freegliboracle 5l1v3r1 junzeliu sailfish009 luan1412167 sunyukuan skylord2 newcodevelop braveryang chenghuige alexjuncpp 00mjk leeseyun

pytorch2keras's Issues

Issue of layer name parsing (self-defined layer) and weight converting

I got a model which has some layers as following:

State dict: ['base.conv1.weight', 'base.bn1.weight', 'base.bn1.bias', 'base.bn1.running_mean', 'base.bn1.running_var', 'base.layer1.0.conv1.weight', ..., 'local_conv_list.0.0.weight', 'local_conv_list.0.0.bias', 'local_conv_list.0.1.weight', 'local_conv_list.0.1.bias', 'local_conv_list.0.1.running_mean', ..., 'fc_list.0.weight', 'fc_list.0.bias', 'fc_list.1.weight', 'fc_list.1.bias', 'fc_list.2.weight', 'fc_list.2.bias', 'fc_list.3.weight', 'fc_list.3.bias', 'fc_list.4.weight', 'fc_list.4.bias', 'fc_list.5.weight', 'fc_list.5.bias']

The definition of the model is : ResNet50 + self.local_conv_list = nn.ModuleList()
where:
self.local_conv_list.append(nn.Sequential(
nn.Conv2d(2048, local_conv_out_channels, 1),
nn.BatchNorm2d(local_conv_out_channels),
nn.ReLU(inplace=True)
))

In converter.py

    node_weights_name = '.'.join(
        re.findall(r'\[([\w\d.]+)\]', node_scope_name)
    )

this function parsing the string, such like:
['PCBModel/ResNet[base]/Sequential[layer4]/Bottleneck[2]/Conv2d[conv3]']
into the original pytorch layer name such as :
base.layer4.2.conv3
then converting the weights accordingly.

But when it comes to the layer which does not have certain naming pattern, such like:
'local_conv_list.0.1.weight',
graph node: PCBModel/Sequential/Conv2d[0]
name in state_dict: 0

the parsing results will be something wrong and it will never matched with any layer name in the State dict, therefore results in the converting error.

So, I got stuck here and was wondering how to handle it, is there anyone could give some constructive suggestion?

Error: InstanceNormalization after ConvTranspose2d

I want to apply InstanceNormalization after nn.ConvTranspose2d. However, an error appeared and it was impossible.
Can not it work with the current program?

Write down the test code and a error.

source code.

import numpy as np
import torch
import torch.nn as nn
from torch.autograd import Variable
from pytorch2keras.converter import pytorch_to_keras
from sys import exit


class TestMultipleInputs(nn.Module):
    """Module for Conv2d conversion testing
    """

    def __init__(self, inp=10, out=16, kernel_size=3, bias=True):
        super(TestMultipleInputs, self).__init__()
        self.conv2d = nn.Conv2d(inp, out, kernel_size=kernel_size, bias=bias)
        self.deconv2d = nn.ConvTranspose2d(inp, out, kernel_size=kernel_size, bias=bias)
        self.in2d = nn.InstanceNorm2d(out)

    def forward(self, x, y, z):
        # return self.in2d(self.conv2d(x)) + self.in2d(self.conv2d(y)) + self.in2d(self.conv2d(z))
        return self.in2d(self.deconv2d(x)) + self.in2d(self.deconv2d(y)) + self.in2d(self.deconv2d(z))
        # return self.conv2d(x) + self.conv2d(y) + self.conv2d(z)


if __name__ == '__main__':
    max_error = 0

    kernel_size = np.random.randint(1, 7)
    inp = np.random.randint(kernel_size + 1, 100)
    out = np.random.randint(1, 100)

    model = TestMultipleInputs(inp, out, kernel_size, inp % 2)

    input_np = np.random.uniform(0, 1, (1, inp, inp, inp))
    input_var = Variable(torch.FloatTensor(input_np))
    input_var2 = Variable(torch.FloatTensor(input_np))
    input_var3 = Variable(torch.FloatTensor(input_np))

    print(input_var.size())
    print(input_var2.size())
    print(input_var3.size())

    output = model(input_var, input_var2, input_var3)

    k_model = pytorch_to_keras(model, [input_var, input_var2, input_var3], [(inp, inp, inp,), (inp, inp, inp,), (inp, inp, inp,)], verbose=True)

    
    pytorch_output = output.data.numpy()
    keras_output = k_model.predict([input_np, input_np, input_np])

an error.

Using TensorFlow backend.
torch.Size([1, 92, 92, 92])
torch.Size([1, 92, 92, 92])
torch.Size([1, 92, 92, 92])
graph(%0 : Float(1, 92, 92, 92)
      %1 : Float(1, 92, 92, 92)
      %2 : Float(1, 92, 92, 92)
      %3 : Float(53, 92, 1, 1)
      %4 : Float(92, 53, 1, 1)) {
  %5 : Float(1, 53, 92, 92) = onnx::ConvTranspose[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%0, %4), scope: TestMultipleInputs/ConvTranspose2d[deconv2d]
  %6 : Dynamic = onnx::Constant[value=<Tensor>](), scope: TestMultipleInputs/InstanceNorm2d[in2d]
  %7 : Dynamic = onnx::Constant[value=<Tensor>](), scope: TestMultipleInputs/InstanceNorm2d[in2d]
  %8 : Float(1, 53, 92, 92) = onnx::InstanceNormalization[epsilon=1e-05](%5, %6, %7), scope: TestMultipleInputs/InstanceNorm2d[in2d]
  %9 : Float(1, 53, 92, 92) = onnx::ConvTranspose[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%1, %4), scope: TestMultipleInputs/ConvTranspose2d[deconv2d]
  %10 : Dynamic = onnx::Constant[value=<Tensor>](), scope: TestMultipleInputs/InstanceNorm2d[in2d]
  %11 : Dynamic = onnx::Constant[value=<Tensor>](), scope: TestMultipleInputs/InstanceNorm2d[in2d]
  %12 : Float(1, 53, 92, 92) = onnx::InstanceNormalization[epsilon=1e-05](%9, %10, %11), scope: TestMultipleInputs/InstanceNorm2d[in2d]
  %13 : Float(1, 53, 92, 92) = onnx::Add(%8, %12), scope: TestMultipleInputs
  %14 : Float(1, 53, 92, 92) = onnx::ConvTranspose[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%2, %4), scope: TestMultipleInputs/ConvTranspose2d[deconv2d]
  %15 : Dynamic = onnx::Constant[value=<Tensor>](), scope: TestMultipleInputs/InstanceNorm2d[in2d]
  %16 : Dynamic = onnx::Constant[value=<Tensor>](), scope: TestMultipleInputs/InstanceNorm2d[in2d]
  %17 : Float(1, 53, 92, 92) = onnx::InstanceNormalization[epsilon=1e-05](%14, %15, %16), scope: TestMultipleInputs/InstanceNorm2d[in2d]
  %18 : Float(1, 53, 92, 92) = onnx::Add(%13, %17), scope: TestMultipleInputs
  return (%18);
}

[18 defined in (%18 : Float(1, 53, 92, 92) = onnx::Add(%13, %17), scope: TestMultipleInputs
)]
Graph outputs: ['18']
State dict: ['conv2d.weight', 'deconv2d.weight']
 ____
graph node: TestMultipleInputs/ConvTranspose2d[deconv2d]
type: onnx::ConvTranspose
inputs: ['input0']
outputs: ['TestMultipleInputs/ConvTranspose2d[deconv2d]']
name in state_dict: deconv2d
attrs: {'dilations': [1, 1], 'group': 1, 'kernel_shape': [1, 1], 'pads': [0, 0, 0, 0], 'strides': [1, 1]}
is_terminal: False
Converting transposed convolution ...
2018-10-20 00:17:50.934191: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
 ____
graph node: TestMultipleInputs/InstanceNorm2d[in2d]
type: onnx::Constant
inputs: []
outputs: ['TestMultipleInputs/InstanceNorm2d[in2d]']
name in state_dict: in2d
attrs: {'value': tensor([ 1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,
         1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,
         1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,
         1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,
         1.,  1.,  1.,  1.,  1.])}
is_terminal: False
Converting constant ...
 ____
graph node: TestMultipleInputs/InstanceNorm2d[in2d]
type: onnx::Constant
inputs: []
outputs: ['TestMultipleInputs/InstanceNorm2d[in2d]']
name in state_dict: in2d
attrs: {'value': tensor([ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.])}
is_terminal: False
Converting constant ...
 ____
graph node: TestMultipleInputs/InstanceNorm2d[in2d]
type: onnx::InstanceNormalization
inputs: ['5', '6', '7']
outputs: ['TestMultipleInputs/InstanceNorm2d[in2d]']
name in state_dict: in2d
attrs: {'epsilon': 1e-05}
is_terminal: False
Converting instancenorm ...
Traceback (most recent call last):
  File "multiple_layer_test.py", line 45, in <module>
    k_model = pytorch_to_keras(model, [input_var, input_var2, input_var3], [(inp, inp, inp,), (inp, inp, inp,), (inp, inp, inp,)], verbose=True)
  File "~/.pyenv/versions/anaconda3-5.1.0/lib/python3.6/site-packages/pytorch2keras/converter.py", line 182, in pytorch_to_keras
    short_names
  File "~/.pyenv/versions/anaconda3-5.1.0/lib/python3.6/site-packages/pytorch2keras/layers.py", line 626, in convert_instancenorm
    layers[scope_name] = lambda_layer(layers[inputs[0]])
  File "~/.pyenv/versions/anaconda3-5.1.0/lib/python3.6/site-packages/keras/engine/base_layer.py", line 457, in __call__
    output = self.call(inputs, **kwargs)
  File "~/.pyenv/versions/anaconda3-5.1.0/lib/python3.6/site-packages/keras/layers/core.py", line 682, in call
    return self.function(inputs, **arguments)
  File "~/.pyenv/versions/anaconda3-5.1.0/lib/python3.6/site-packages/pytorch2keras/layers.py", line 622, in target_layer
    trainable=False)
  File "~/.pyenv/versions/anaconda3-5.1.0/lib/python3.6/site-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 183, in func_with_args
    return func(*args, **current_args)
  File "~/.pyenv/versions/anaconda3-5.1.0/lib/python3.6/site-packages/tensorflow/contrib/layers/python/layers/normalization.py", line 120, in instance_norm
    inputs.name, params_shape))
ValueError: Inputs deconv2d0.27781545017061593/transpose_1:0 has undefined channels dimension (?,).

In my environment.

PyTorch 0.4.0
Keras 2.2.2
Tensorflow 1.9.0

It's not working with Pytorch = 1.0

I've downgraded to 0.4.1 and it worked

KeyError: 'Gather' while converting model

Hi again!
Another problem with converting model to keras, if we have input as LongTensor of indices for nn.Embedding (not one-hot encoded input like in issue 11).

This is simple code for reproducing:

from torch import nn, zeros, LongTensor, FloatTensor, onnx
from torch.autograd import Variable
import torch.nn.functional as F
from converter import pytorch_to_keras

class SimpleNet(nn.Module):
    def __init__(self, input_size):
        super(SimpleNet, self).__init__()
        
        self.embedd = nn.Embedding(input_size, 100)
        self.fc1 = nn.Linear(100, 200)
        self.fc2 = nn.Linear(200, 200)
        self.fc3 = nn.Linear(200, 300)
                
    def forward(self, input):
        r_0 = self.embedd(input).sum(dim=1)
        r_1 = F.tanh(self.fc1(r_0).div(10.))
        r_2 = F.tanh(self.fc2(r_1).div(10.)) + r_1
        r_3 = F.tanh(self.fc3(r_2).div(10.))
        return r_3

input = Variable(LongTensor([[1,2,4,5]]))

simple_net = SimpleNet(1000)
out = simple_net(input)

k_model = pytorch_to_keras(simple_net, input, (1, 4), verbose=True)

And this is the output:

graph(%0 : Long(1, 4)
      %1 : Float(1000, 100)
      %2 : Float(200, 100)
      %3 : Float(200)
      %4 : Float(200, 200)
      %5 : Float(200)
      %6 : Float(300, 200)
      %7 : Float(300)) {
  %8 : Float(1, 4, 100) = Gather(%1, %0), scope: SimpleNet/Embedding[embedd]
  %9 : Float(1, 100) = ReduceSum[axes=[1], keepdims=0](%8), scope: SimpleNet
  %10 : Float(1, 200) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%9, %2, %3), scope: SimpleNet/Linear[fc1]
  %11 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
  %12 : Float(1, 200) = Div[broadcast=1](%10, %11), scope: SimpleNet
  %13 : Float(1, 200) = Tanh(%12), scope: SimpleNet
  %14 : Float(1, 200) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%13, %4, %5), scope: SimpleNet/Linear[fc2]
  %15 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
  %16 : Float(1, 200) = Div[broadcast=1](%14, %15), scope: SimpleNet
  %17 : Float(1, 200) = Tanh(%16), scope: SimpleNet
  %18 : Float(1, 200) = Add(%17, %13), scope: SimpleNet
  %19 : Float(1, 300) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%18, %6, %7), scope: SimpleNet/Linear[fc3]
  %20 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
  %21 : Float(1, 300) = Div[broadcast=1](%19, %20), scope: SimpleNet
  %22 : Float(1, 300) = Tanh(%21), scope: SimpleNet
  return (%22);
}

[22 defined in (%22 : Float(1, 300) = Tanh(%21), scope: SimpleNet
)]
['embedd.weight', 'fc1.weight', 'fc1.bias', 'fc2.weight', 'fc2.bias', 'fc3.weight', 'fc3.bias']
 ____ 
graph node: SimpleNet/Embedding[embedd]
type: Gather
inputs: ['input']
outputs: ['SimpleNet/Embedding[embedd]']
name in state_dict: embedd
attrs: {}
is_terminal: False
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-43-20307fc45f89> in <module>()
----> 1 k_model = pytorch_to_keras(simple_net, input, (1, 4), verbose=True)  #we should specify shape of the input tensor

~/workspace/pytorch2keras/pytorch2keras/converter.py in pytorch_to_keras(model, args, input_shape, change_ordering, training, verbose)
    154             print('attrs:', node_attrs)
    155             print('is_terminal:', node_id in graph_outputs)
--> 156         AVAILABLE_CONVERTERS[node_type](
    157             node_attrs,
    158             node_weights_name, node_id,

KeyError: 'Gather'

So, please, could you add Gather also to AVAILABLE_CONVERTERS ?
thanks

Doesn't support PixelShuffle layers!

Describe the bug
I am trying to convert a pytorch model that has a PixelShuffle layer.
The layer is converted as a Lambda layer, but unfortunately, it doesn't work properly.
Please note the model summary below where the PixelShuffle is supposed to get a (None, 256, 256, 256) tensor as its input and output a (None, 64, 512, 512) tensor. The PixelShuffle is translated to three steps (Lambda-Permute-Lambda). However, for some reason, the channels are manipulated (moved to the middle, last and then first again), which distorts the output image. Please note that the "image_data_format" of the keras is "channels_first".

part of keras model summary:

input image: (3, 256, 256)

output image: (3, 512, 512):

To Reproduce
The model that I am trying to convert is from this repo:
https://github.com/nmhkahn/CARN-pytorch

To reproduce the workflow, please substitute the main function in sample.py with:

def main(cfg):
    
    module = importlib.import_module("model.{}".format(cfg.model))  #it imports model.carn.py
    net = module.Net(multi_scale=True, group = cfg.group)
    print(json.dumps(vars(cfg), indent=4, sort_keys=True))
    
    #state_dict = torch.load(cfg.ckpt_path)
    state_dict = torch.load(cfg.ckpt_path, map_location='cpu')
    new_state_dict = OrderedDict()
    for k, v in state_dict.items():
        name = k
        new_state_dict[name] = v
    net.load_state_dict(new_state_dict)
    
    scale = cfg.scale
    # device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    device = torch.device("cpu")
    net = net.to(device)
    model = net
    print(model)
    
    # keras
    input_np = np.random.uniform(0, 1, (1, 3, 256, 256))
    input_var = Variable(torch.FloatTensor(input_np))    
    k_model = pytorch_to_keras(model, input_var, [(3, 256, 256,)], verbose=True)
    k_model.summary(line_length=170)          
            
    input_img = '../data/img_001_SRF_2_LR.png'
    img = Image.open(input_img).convert("RGB")
    image = np.array(img, dtype=np.float32)[...,:3]
    image = np.rollaxis(image, 2, 0)         # channel first
    image = image.astype(np.float32) / 255   # normalization

    input_image = np.expand_dims(image, axis=0)
    print("Image 1 shape: ", input_image.shape)
    
    prediction = k_model.predict(input_image)
    prediction = prediction[0]
    print("prediction: ", prediction.shape)
    
    prediction = np.rollaxis(prediction, 1, 0)
    prediction = np.rollaxis(prediction, 2, 1)
    prediction = prediction * 255            # denormalization
    print("prediction: ", prediction.shape)
    
    
    output_path = '../data/img_001_SRF_2_SR_keras.png'
    print("min", np.min(prediction))
    print("max", np.max(prediction))
    result = Image.fromarray((prediction).astype(np.uint8))
    result.save(output_path)
    #k_model.save('my_model.h5')

Expected behavior
We are looking for a super-sesolution of the input image. The conversion of the PixelShuffle layer is supposed to get a (None, 256, 256, 256) tensor as its input and output a (None, 64, 512, 512). That is, channel will be divided by 4 and the width and height multiplied by 2.

Do you have any plan to support slicing (or narrow layer) layer?

I was converting a model which using ResNet50 as backbone following by some slicing and avg pool and conv operation, but the problem is that it seems the "onnx::Slice" layer is not supported in the current version therefore results in the converting error. So I'd like to know if you are going to add the functionality for converting the slice layer?

error when test squeezenext in the tests folder

hi @nerox8664
i need your help please
i ran the python file of squeezenext in the tests folder (my machine includes pytorch 0.4.1 - keras 2.2.0 and tensorflow 1.9.0) but it gave me the following error:

TypeError Traceback (most recent call last)
in ()
395 output = model(input_var)
396
--> 397 k_model = pytorch_to_keras(model, input_var, (3, 224, 224,), verbose=True)
398
399 pytorch_output = output.data.numpy()

~\Downloads\Compressed\pytorch2keras-master\pytorch2keras\converter.py in pytorch_to_keras(model, args, input_shapes, change_ordering, training, verbose, short_names)
96
97 # _optimize_trace(trace, False)
---> 98 trace.set_graph(_optimize_graph(trace.graph(), False))
99
100 if verbose:

~\Downloads\Compressed\pytorch2keras-master\pytorch2keras\converter.py in _optimize_graph(graph, aten)
41 torch._C._jit_pass_peephole(graph)
42 torch._C._jit_pass_lint(graph)
---> 43 graph = torch._C._jit_pass_onnx(graph, aten)
44 torch._C._jit_pass_lint(graph)
45 torch._C._jit_pass_onnx_peephole(graph)

TypeError: _jit_pass_onnx(): incompatible function arguments. The following argument types are supported:
1. (arg0: torch::jit::Graph, arg1: torch._C._onnx.OperatorExportTypes) -> torch::jit::Graph

Invoked with: graph(%0 : Float(1, 3, 224, 224)
%1 : Float(64, 3, 7, 7)
%2 : Float(64)
%3 : Float(64)
%4 : Float(64)
%5 : Float(64)
%6 : Float(64)
%7 : Long()
%8 : Float(16, 64, 1, 1)
%9 : Float(16)
%10 : Float(16)
%11 : Float(16)
%12 : Float(16)
%13 : Float(16)
%14 : Long()
%15 : Float(8, 16, 1, 1)
%16 : Float(8)
%17 : Float(8)
%18 : Float(8)
%19 : Float(8)
%20 : Float(8)
%21 : Long()
%22 : Float(16, 8, 1, 3)
%23 : Float(16)
%24 : Float(16)
%25 : Float(16)
%26 : Float(16)
%27 : Float(16)
%28 : Long()
%29 : Float(16, 16, 3, 1)
%30 : Float(16)
%31 : Float(16)
%32 : Float(16)
%33 : Float(16)
%34 : Float(16)
%35 : Long()
%36 : Float(32, 16, 1, 1)
%37 : Float(32)
%38 : Float(32)
%39 : Float(32)
%40 : Float(32)
%41 : Float(32)
%42 : Long()
%43 : Float(32, 64, 1, 1)
%44 : Float(32)
%45 : Float(32)
%46 : Float(32)
%47 : Float(32)
%48 : Float(32)
%49 : Long()
%50 : Float(16, 32, 1, 1)
%51 : Float(16)
%52 : Float(16)
%53 : Float(16)
%54 : Float(16)
%55 : Float(16)
%56 : Long()
%57 : Float(8, 16, 1, 1)
%58 : Float(8)
%59 : Float(8)
%60 : Float(8)
%61 : Float(8)
%62 : Float(8)
%63 : Long()
%64 : Float(16, 8, 1, 3)
%65 : Float(16)
%66 : Float(16)
%67 : Float(16)
%68 : Float(16)
%69 : Float(16)
%70 : Long()
%71 : Float(16, 16, 3, 1)
%72 : Float(16)
%73 : Float(16)
%74 : Float(16)
%75 : Float(16)
%76 : Float(16)
%77 : Long()
%78 : Float(32, 16, 1, 1)
%79 : Float(32)
%80 : Float(32)
%81 : Float(32)
%82 : Float(32)
%83 : Float(32)
%84 : Long()
%85 : Float(16, 32, 1, 1)
%86 : Float(16)
%87 : Float(16)
%88 : Float(16)
%89 : Float(16)
%90 : Float(16)
%91 : Long()
%92 : Float(8, 16, 1, 1)
%93 : Float(8)
%94 : Float(8)
%95 : Float(8)
%96 : Float(8)
%97 : Float(8)
%98 : Long()
%99 : Float(16, 8, 1, 3)
%100 : Float(16)
%101 : Float(16)
%102 : Float(16)
%103 : Float(16)
%104 : Float(16)
%105 : Long()
%106 : Float(16, 16, 3, 1)
%107 : Float(16)
%108 : Float(16)
%109 : Float(16)
%110 : Float(16)
%111 : Float(16)
%112 : Long()
%113 : Float(32, 16, 1, 1)
%114 : Float(32)
%115 : Float(32)
%116 : Float(32)
%117 : Float(32)
%118 : Float(32)
%119 : Long()
%120 : Float(16, 32, 1, 1)
%121 : Float(16)
%122 : Float(16)
%123 : Float(16)
%124 : Float(16)
%125 : Float(16)
%126 : Long()
%127 : Float(8, 16, 1, 1)
%128 : Float(8)
%129 : Float(8)
%130 : Float(8)
%131 : Float(8)
%132 : Float(8)
%133 : Long()
%134 : Float(16, 8, 1, 3)
%135 : Float(16)
%136 : Float(16)
%137 : Float(16)
%138 : Float(16)
%139 : Float(16)
%140 : Long()
%141 : Float(16, 16, 3, 1)
%142 : Float(16)
%143 : Float(16)
%144 : Float(16)
%145 : Float(16)
%146 : Float(16)
%147 : Long()
%148 : Float(32, 16, 1, 1)
%149 : Float(32)
%150 : Float(32)
%151 : Float(32)
%152 : Float(32)
%153 : Float(32)
%154 : Long()
%155 : Float(16, 32, 1, 1)
%156 : Float(16)
%157 : Float(16)
%158 : Float(16)
%159 : Float(16)
%160 : Float(16)
%161 : Long()
%162 : Float(8, 16, 1, 1)
%163 : Float(8)
%164 : Float(8)
%165 : Float(8)
%166 : Float(8)
%167 : Float(8)
%168 : Long()
%169 : Float(16, 8, 1, 3)
%170 : Float(16)
%171 : Float(16)
%172 : Float(16)
%173 : Float(16)
%174 : Float(16)
%175 : Long()
%176 : Float(16, 16, 3, 1)
%177 : Float(16)
%178 : Float(16)
%179 : Float(16)
%180 : Float(16)
%181 : Float(16)
%182 : Long()
%183 : Float(32, 16, 1, 1)
%184 : Float(32)
%185 : Float(32)
%186 : Float(32)
%187 : Float(32)
%188 : Float(32)
%189 : Long()
%190 : Float(16, 32, 1, 1)
%191 : Float(16)
%192 : Float(16)
%193 : Float(16)
%194 : Float(16)
%195 : Float(16)
%196 : Long()
%197 : Float(8, 16, 1, 1)
%198 : Float(8)
%199 : Float(8)
%200 : Float(8)
%201 : Float(8)
%202 : Float(8)
%203 : Long()
%204 : Float(16, 8, 1, 3)
%205 : Float(16)
%206 : Float(16)
%207 : Float(16)
%208 : Float(16)
%209 : Float(16)
%210 : Long()
%211 : Float(16, 16, 3, 1)
%212 : Float(16)
%213 : Float(16)
%214 : Float(16)
%215 : Float(16)
%216 : Float(16)
%217 : Long()
%218 : Float(32, 16, 1, 1)
%219 : Float(32)
%220 : Float(32)
%221 : Float(32)
%222 : Float(32)
%223 : Float(32)
%224 : Long()
%225 : Float(32, 32, 1, 1)
%226 : Float(32)
%227 : Float(32)
%228 : Float(32)
%229 : Float(32)
%230 : Float(32)
%231 : Long()
%232 : Float(16, 32, 1, 1)
%233 : Float(16)
%234 : Float(16)
%235 : Float(16)
%236 : Float(16)
%237 : Float(16)
%238 : Long()
%239 : Float(32, 16, 1, 3)
%240 : Float(32)
%241 : Float(32)
%242 : Float(32)
%243 : Float(32)
%244 : Float(32)
%245 : Long()
%246 : Float(32, 32, 3, 1)
%247 : Float(32)
%248 : Float(32)
%249 : Float(32)
%250 : Float(32)
%251 : Float(32)
%252 : Long()
%253 : Float(64, 32, 1, 1)
%254 : Float(64)
%255 : Float(64)
%256 : Float(64)
%257 : Float(64)
%258 : Float(64)
%259 : Long()
%260 : Float(64, 32, 1, 1)
%261 : Float(64)
%262 : Float(64)
%263 : Float(64)
%264 : Float(64)
%265 : Float(64)
%266 : Long()
%267 : Float(32, 64, 1, 1)
%268 : Float(32)
%269 : Float(32)
%270 : Float(32)
%271 : Float(32)
%272 : Float(32)
%273 : Long()
%274 : Float(16, 32, 1, 1)
%275 : Float(16)
%276 : Float(16)
%277 : Float(16)
%278 : Float(16)
%279 : Float(16)
%280 : Long()
%281 : Float(32, 16, 1, 3)
%282 : Float(32)
%283 : Float(32)
%284 : Float(32)
%285 : Float(32)
%286 : Float(32)
%287 : Long()
%288 : Float(32, 32, 3, 1)
%289 : Float(32)
%290 : Float(32)
%291 : Float(32)
%292 : Float(32)
%293 : Float(32)
%294 : Long()
%295 : Float(64, 32, 1, 1)
%296 : Float(64)
%297 : Float(64)
%298 : Float(64)
%299 : Float(64)
%300 : Float(64)
%301 : Long()
%302 : Float(32, 64, 1, 1)
%303 : Float(32)
%304 : Float(32)
%305 : Float(32)
%306 : Float(32)
%307 : Float(32)
%308 : Long()
%309 : Float(16, 32, 1, 1)
%310 : Float(16)
%311 : Float(16)
%312 : Float(16)
%313 : Float(16)
%314 : Float(16)
%315 : Long()
%316 : Float(32, 16, 1, 3)
%317 : Float(32)
%318 : Float(32)
%319 : Float(32)
%320 : Float(32)
%321 : Float(32)
%322 : Long()
%323 : Float(32, 32, 3, 1)
%324 : Float(32)
%325 : Float(32)
%326 : Float(32)
%327 : Float(32)
%328 : Float(32)
%329 : Long()
%330 : Float(64, 32, 1, 1)
%331 : Float(64)
%332 : Float(64)
%333 : Float(64)
%334 : Float(64)
%335 : Float(64)
%336 : Long()
%337 : Float(32, 64, 1, 1)
%338 : Float(32)
%339 : Float(32)
%340 : Float(32)
%341 : Float(32)
%342 : Float(32)
%343 : Long()
%344 : Float(16, 32, 1, 1)
%345 : Float(16)
%346 : Float(16)
%347 : Float(16)
%348 : Float(16)
%349 : Float(16)
%350 : Long()
%351 : Float(32, 16, 1, 3)
%352 : Float(32)
%353 : Float(32)
%354 : Float(32)
%355 : Float(32)
%356 : Float(32)
%357 : Long()
%358 : Float(32, 32, 3, 1)
%359 : Float(32)
%360 : Float(32)
%361 : Float(32)
%362 : Float(32)
%363 : Float(32)
%364 : Long()
%365 : Float(64, 32, 1, 1)
%366 : Float(64)
%367 : Float(64)
%368 : Float(64)
%369 : Float(64)
%370 : Float(64)
%371 : Long()
%372 : Float(32, 64, 1, 1)
%373 : Float(32)
%374 : Float(32)
%375 : Float(32)
%376 : Float(32)
%377 : Float(32)
%378 : Long()
%379 : Float(16, 32, 1, 1)
%380 : Float(16)
%381 : Float(16)
%382 : Float(16)
%383 : Float(16)
%384 : Float(16)
%385 : Long()
%386 : Float(32, 16, 1, 3)
%387 : Float(32)
%388 : Float(32)
%389 : Float(32)
%390 : Float(32)
%391 : Float(32)
%392 : Long()
%393 : Float(32, 32, 3, 1)
%394 : Float(32)
%395 : Float(32)
%396 : Float(32)
%397 : Float(32)
%398 : Float(32)
%399 : Long()
%400 : Float(64, 32, 1, 1)
%401 : Float(64)
%402 : Float(64)
%403 : Float(64)
%404 : Float(64)
%405 : Float(64)
%406 : Long()
%407 : Float(32, 64, 1, 1)
%408 : Float(32)
%409 : Float(32)
%410 : Float(32)
%411 : Float(32)
%412 : Float(32)
%413 : Long()
%414 : Float(16, 32, 1, 1)
%415 : Float(16)
%416 : Float(16)
%417 : Float(16)
%418 : Float(16)
%419 : Float(16)
%420 : Long()
%421 : Float(32, 16, 1, 3)
%422 : Float(32)
%423 : Float(32)
%424 : Float(32)
%425 : Float(32)
%426 : Float(32)
%427 : Long()
%428 : Float(32, 32, 3, 1)
%429 : Float(32)
%430 : Float(32)
%431 : Float(32)
%432 : Float(32)
%433 : Float(32)
%434 : Long()
%435 : Float(64, 32, 1, 1)
%436 : Float(64)
%437 : Float(64)
%438 : Float(64)
%439 : Float(64)
%440 : Float(64)
%441 : Long()
%442 : Float(64, 64, 1, 1)
%443 : Float(64)
%444 : Float(64)
%445 : Float(64)
%446 : Float(64)
%447 : Float(64)
%448 : Long()
%449 : Float(32, 64, 1, 1)
%450 : Float(32)
%451 : Float(32)
%452 : Float(32)
%453 : Float(32)
%454 : Float(32)
%455 : Long()
%456 : Float(64, 32, 1, 3)
%457 : Float(64)
%458 : Float(64)
%459 : Float(64)
%460 : Float(64)
%461 : Float(64)
%462 : Long()
%463 : Float(64, 64, 3, 1)
%464 : Float(64)
%465 : Float(64)
%466 : Float(64)
%467 : Float(64)
%468 : Float(64)
%469 : Long()
%470 : Float(128, 64, 1, 1)
%471 : Float(128)
%472 : Float(128)
%473 : Float(128)
%474 : Float(128)
%475 : Float(128)
%476 : Long()
%477 : Float(128, 64, 1, 1)
%478 : Float(128)
%479 : Float(128)
%480 : Float(128)
%481 : Float(128)
%482 : Float(128)
%483 : Long()
%484 : Float(64, 128, 1, 1)
%485 : Float(64)
%486 : Float(64)
%487 : Float(64)
%488 : Float(64)
%489 : Float(64)
%490 : Long()
%491 : Float(32, 64, 1, 1)
%492 : Float(32)
%493 : Float(32)
%494 : Float(32)
%495 : Float(32)
%496 : Float(32)
%497 : Long()
%498 : Float(64, 32, 1, 3)
%499 : Float(64)
%500 : Float(64)
%501 : Float(64)
%502 : Float(64)
%503 : Float(64)
%504 : Long()
%505 : Float(64, 64, 3, 1)
%506 : Float(64)
%507 : Float(64)
%508 : Float(64)
%509 : Float(64)
%510 : Float(64)
%511 : Long()
%512 : Float(128, 64, 1, 1)
%513 : Float(128)
%514 : Float(128)
%515 : Float(128)
%516 : Float(128)
%517 : Float(128)
%518 : Long()
%519 : Float(64, 128, 1, 1)
%520 : Float(64)
%521 : Float(64)
%522 : Float(64)
%523 : Float(64)
%524 : Float(64)
%525 : Long()
%526 : Float(32, 64, 1, 1)
%527 : Float(32)
%528 : Float(32)
%529 : Float(32)
%530 : Float(32)
%531 : Float(32)
%532 : Long()
%533 : Float(64, 32, 1, 3)
%534 : Float(64)
%535 : Float(64)
%536 : Float(64)
%537 : Float(64)
%538 : Float(64)
%539 : Long()
%540 : Float(64, 64, 3, 1)
%541 : Float(64)
%542 : Float(64)
%543 : Float(64)
%544 : Float(64)
%545 : Float(64)
%546 : Long()
%547 : Float(128, 64, 1, 1)
%548 : Float(128)
%549 : Float(128)
%550 : Float(128)
%551 : Float(128)
%552 : Float(128)
%553 : Long()
%554 : Float(64, 128, 1, 1)
%555 : Float(64)
%556 : Float(64)
%557 : Float(64)
%558 : Float(64)
%559 : Float(64)
%560 : Long()
%561 : Float(32, 64, 1, 1)
%562 : Float(32)
%563 : Float(32)
%564 : Float(32)
%565 : Float(32)
%566 : Float(32)
%567 : Long()
%568 : Float(64, 32, 1, 3)
%569 : Float(64)
%570 : Float(64)
%571 : Float(64)
%572 : Float(64)
%573 : Float(64)
%574 : Long()
%575 : Float(64, 64, 3, 1)
%576 : Float(64)
%577 : Float(64)
%578 : Float(64)
%579 : Float(64)
%580 : Float(64)
%581 : Long()
%582 : Float(128, 64, 1, 1)
%583 : Float(128)
%584 : Float(128)
%585 : Float(128)
%586 : Float(128)
%587 : Float(128)
%588 : Long()
%589 : Float(64, 128, 1, 1)
%590 : Float(64)
%591 : Float(64)
%592 : Float(64)
%593 : Float(64)
%594 : Float(64)
%595 : Long()
%596 : Float(32, 64, 1, 1)
%597 : Float(32)
%598 : Float(32)
%599 : Float(32)
%600 : Float(32)
%601 : Float(32)
%602 : Long()
%603 : Float(64, 32, 1, 3)
%604 : Float(64)
%605 : Float(64)
%606 : Float(64)
%607 : Float(64)
%608 : Float(64)
%609 : Long()
%610 : Float(64, 64, 3, 1)
%611 : Float(64)
%612 : Float(64)
%613 : Float(64)
%614 : Float(64)
%615 : Float(64)
%616 : Long()
%617 : Float(128, 64, 1, 1)
%618 : Float(128)
%619 : Float(128)
%620 : Float(128)
%621 : Float(128)
%622 : Float(128)
%623 : Long()
%624 : Float(64, 128, 1, 1)
%625 : Float(64)
%626 : Float(64)
%627 : Float(64)
%628 : Float(64)
%629 : Float(64)
%630 : Long()
%631 : Float(32, 64, 1, 1)
%632 : Float(32)
%633 : Float(32)
%634 : Float(32)
%635 : Float(32)
%636 : Float(32)
%637 : Long()
%638 : Float(64, 32, 1, 3)
%639 : Float(64)
%640 : Float(64)
%641 : Float(64)
%642 : Float(64)
%643 : Float(64)
%644 : Long()
%645 : Float(64, 64, 3, 1)
%646 : Float(64)
%647 : Float(64)
%648 : Float(64)
%649 : Float(64)
%650 : Float(64)
%651 : Long()
%652 : Float(128, 64, 1, 1)
%653 : Float(128)
%654 : Float(128)
%655 : Float(128)
%656 : Float(128)
%657 : Float(128)
%658 : Long()
%659 : Float(64, 128, 1, 1)
%660 : Float(64)
%661 : Float(64)
%662 : Float(64)
%663 : Float(64)
%664 : Float(64)
%665 : Long()
%666 : Float(32, 64, 1, 1)
%667 : Float(32)
%668 : Float(32)
%669 : Float(32)
%670 : Float(32)
%671 : Float(32)
%672 : Long()
%673 : Float(64, 32, 1, 3)
%674 : Float(64)
%675 : Float(64)
%676 : Float(64)
%677 : Float(64)
%678 : Float(64)
%679 : Long()
%680 : Float(64, 64, 3, 1)
%681 : Float(64)
%682 : Float(64)
%683 : Float(64)
%684 : Float(64)
%685 : Float(64)
%686 : Long()
%687 : Float(128, 64, 1, 1)
%688 : Float(128)
%689 : Float(128)
%690 : Float(128)
%691 : Float(128)
%692 : Float(128)
%693 : Long()
%694 : Float(64, 128, 1, 1)
%695 : Float(64)
%696 : Float(64)
%697 : Float(64)
%698 : Float(64)
%699 : Float(64)
%700 : Long()
%701 : Float(32, 64, 1, 1)
%702 : Float(32)
%703 : Float(32)
%704 : Float(32)
%705 : Float(32)
%706 : Float(32)
%707 : Long()
%708 : Float(64, 32, 1, 3)
%709 : Float(64)
%710 : Float(64)
%711 : Float(64)
%712 : Float(64)
%713 : Float(64)
%714 : Long()
%715 : Float(64, 64, 3, 1)
%716 : Float(64)
%717 : Float(64)
%718 : Float(64)
%719 : Float(64)
%720 : Float(64)
%721 : Long()
%722 : Float(128, 64, 1, 1)
%723 : Float(128)
%724 : Float(128)
%725 : Float(128)
%726 : Float(128)
%727 : Float(128)
%728 : Long()
%729 : Float(128, 128, 1, 1)
%730 : Float(128)
%731 : Float(128)
%732 : Float(128)
%733 : Float(128)
%734 : Float(128)
%735 : Long()
%736 : Float(64, 128, 1, 1)
%737 : Float(64)
%738 : Float(64)
%739 : Float(64)
%740 : Float(64)
%741 : Float(64)
%742 : Long()
%743 : Float(128, 64, 1, 3)
%744 : Float(128)
%745 : Float(128)
%746 : Float(128)
%747 : Float(128)
%748 : Float(128)
%749 : Long()
%750 : Float(128, 128, 3, 1)
%751 : Float(128)
%752 : Float(128)
%753 : Float(128)
%754 : Float(128)
%755 : Float(128)
%756 : Long()
%757 : Float(256, 128, 1, 1)
%758 : Float(256)
%759 : Float(256)
%760 : Float(256)
%761 : Float(256)
%762 : Float(256)
%763 : Long()
%764 : Float(256, 128, 1, 1)
%765 : Float(256)
%766 : Float(256)
%767 : Float(256)
%768 : Float(256)
%769 : Float(256)
%770 : Long()
%771 : Float(128, 256, 1, 1)
%772 : Float(128)
%773 : Float(128)
%774 : Float(128)
%775 : Float(128)
%776 : Float(128)
%777 : Long()
%778 : Float(1000, 128)
%779 : Float(1000)) {
%785 : Float(1, 64, 110, 110) = aten::_convolution[stride=[2, 2], padding=[1, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%0, %1, %2), scope: SqueezeNext/Sequential[features]/SqnxtInitBlock[init_block]/SqnxtConv[conv]/Conv2d[conv]
%790 : Float(1, 64, 110, 110) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%785, %3, %4, %5, %6), scope: SqueezeNext/Sequential[features]/SqnxtInitBlock[init_block]/SqnxtConv[conv]/BatchNorm2d[bn]
%792 : Float(1, 64, 110, 110) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/SqnxtInitBlock[init_block]/SqnxtConv[conv]/ReLU[activ]
%795 : Float(1, 64, 55, 55), %796 : Long(1, 64, 55, 55) = aten::max_pool2d_with_indiceskernel_size=[3, 3], stride=[2, 2], padding=[0, 0], dilation=[1, 1], ceil_mode=1, scope: SqueezeNext/Sequential[features]/SqnxtInitBlock[init_block]/MaxPool2d[pool]
%802 : Float(1, 32, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%795, %43, %44), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/Conv2d[conv]
%807 : Float(1, 32, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%802, %45, %46, %47, %48), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/BatchNorm2d[bn]
%809 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/ReLU[activ]
%811 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/ReLU[activ]
%817 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%795, %8, %9), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv1]/Conv2d[conv]
%822 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%817, %10, %11, %12, %13), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv1]/BatchNorm2d[bn]
%824 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv1]/ReLU[activ]
%830 : Float(1, 8, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%824, %15, %16), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv2]/Conv2d[conv]
%835 : Float(1, 8, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%830, %17, %18, %19, %20), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv2]/BatchNorm2d[bn]
%837 : Float(1, 8, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv2]/ReLU[activ]
%843 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%837, %22, %23), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv3]/Conv2d[conv]
%848 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%843, %24, %25, %26, %27), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv3]/BatchNorm2d[bn]
%850 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv3]/ReLU[activ]
%856 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%850, %29, %30), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv4]/Conv2d[conv]
%861 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%856, %31, %32, %33, %34), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv4]/BatchNorm2d[bn]
%863 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv4]/ReLU[activ]
%869 : Float(1, 32, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%863, %36, %37), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv5]/Conv2d[conv]
%874 : Float(1, 32, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%869, %38, %39, %40, %41), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv5]/BatchNorm2d[bn]
%876 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/SqnxtConv[conv5]/ReLU[activ]
%877 : Float(1, 32, 55, 55) = aten::add[alpha={1}](%876, %811), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]
%879 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit1]/ReLU[activ]
%881 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/ReLU[activ]
%887 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%881, %50, %51), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv1]/Conv2d[conv]
%892 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%887, %52, %53, %54, %55), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv1]/BatchNorm2d[bn]
%894 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv1]/ReLU[activ]
%900 : Float(1, 8, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%894, %57, %58), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv2]/Conv2d[conv]
%905 : Float(1, 8, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%900, %59, %60, %61, %62), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv2]/BatchNorm2d[bn]
%907 : Float(1, 8, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv2]/ReLU[activ]
%913 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%907, %64, %65), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv3]/Conv2d[conv]
%918 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%913, %66, %67, %68, %69), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv3]/BatchNorm2d[bn]
%920 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv3]/ReLU[activ]
%926 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%920, %71, %72), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv4]/Conv2d[conv]
%931 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%926, %73, %74, %75, %76), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv4]/BatchNorm2d[bn]
%933 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv4]/ReLU[activ]
%939 : Float(1, 32, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%933, %78, %79), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv5]/Conv2d[conv]
%944 : Float(1, 32, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%939, %80, %81, %82, %83), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv5]/BatchNorm2d[bn]
%946 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/SqnxtConv[conv5]/ReLU[activ]
%947 : Float(1, 32, 55, 55) = aten::add[alpha={1}](%946, %881), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]
%949 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit2]/ReLU[activ]
%951 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/ReLU[activ]
%957 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%951, %85, %86), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv1]/Conv2d[conv]
%962 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%957, %87, %88, %89, %90), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv1]/BatchNorm2d[bn]
%964 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv1]/ReLU[activ]
%970 : Float(1, 8, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%964, %92, %93), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv2]/Conv2d[conv]
%975 : Float(1, 8, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%970, %94, %95, %96, %97), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv2]/BatchNorm2d[bn]
%977 : Float(1, 8, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv2]/ReLU[activ]
%983 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%977, %99, %100), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv3]/Conv2d[conv]
%988 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%983, %101, %102, %103, %104), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv3]/BatchNorm2d[bn]
%990 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv3]/ReLU[activ]
%996 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%990, %106, %107), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv4]/Conv2d[conv]
%1001 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%996, %108, %109, %110, %111), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1003 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv4]/ReLU[activ]
%1009 : Float(1, 32, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1003, %113, %114), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv5]/Conv2d[conv]
%1014 : Float(1, 32, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1009, %115, %116, %117, %118), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1016 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/SqnxtConv[conv5]/ReLU[activ]
%1017 : Float(1, 32, 55, 55) = aten::add[alpha={1}](%1016, %951), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]
%1019 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit3]/ReLU[activ]
%1021 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/ReLU[activ]
%1027 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1021, %120, %121), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv1]/Conv2d[conv]
%1032 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1027, %122, %123, %124, %125), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1034 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv1]/ReLU[activ]
%1040 : Float(1, 8, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1034, %127, %128), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv2]/Conv2d[conv]
%1045 : Float(1, 8, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1040, %129, %130, %131, %132), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1047 : Float(1, 8, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv2]/ReLU[activ]
%1053 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1047, %134, %135), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv3]/Conv2d[conv]
%1058 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1053, %136, %137, %138, %139), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1060 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv3]/ReLU[activ]
%1066 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1060, %141, %142), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv4]/Conv2d[conv]
%1071 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1066, %143, %144, %145, %146), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1073 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv4]/ReLU[activ]
%1079 : Float(1, 32, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1073, %148, %149), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv5]/Conv2d[conv]
%1084 : Float(1, 32, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1079, %150, %151, %152, %153), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1086 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/SqnxtConv[conv5]/ReLU[activ]
%1087 : Float(1, 32, 55, 55) = aten::add[alpha={1}](%1086, %1021), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]
%1089 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit4]/ReLU[activ]
%1091 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/ReLU[activ]
%1097 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1091, %155, %156), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv1]/Conv2d[conv]
%1102 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1097, %157, %158, %159, %160), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1104 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv1]/ReLU[activ]
%1110 : Float(1, 8, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1104, %162, %163), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv2]/Conv2d[conv]
%1115 : Float(1, 8, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1110, %164, %165, %166, %167), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1117 : Float(1, 8, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv2]/ReLU[activ]
%1123 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1117, %169, %170), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv3]/Conv2d[conv]
%1128 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1123, %171, %172, %173, %174), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1130 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv3]/ReLU[activ]
%1136 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1130, %176, %177), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv4]/Conv2d[conv]
%1141 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1136, %178, %179, %180, %181), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1143 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv4]/ReLU[activ]
%1149 : Float(1, 32, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1143, %183, %184), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv5]/Conv2d[conv]
%1154 : Float(1, 32, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1149, %185, %186, %187, %188), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1156 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/SqnxtConv[conv5]/ReLU[activ]
%1157 : Float(1, 32, 55, 55) = aten::add[alpha={1}](%1156, %1091), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]
%1159 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit5]/ReLU[activ]
%1161 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/ReLU[activ]
%1167 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1161, %190, %191), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv1]/Conv2d[conv]
%1172 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1167, %192, %193, %194, %195), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1174 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv1]/ReLU[activ]
%1180 : Float(1, 8, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1174, %197, %198), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv2]/Conv2d[conv]
%1185 : Float(1, 8, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1180, %199, %200, %201, %202), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1187 : Float(1, 8, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv2]/ReLU[activ]
%1193 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1187, %204, %205), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv3]/Conv2d[conv]
%1198 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1193, %206, %207, %208, %209), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1200 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv3]/ReLU[activ]
%1206 : Float(1, 16, 55, 55) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1200, %211, %212), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv4]/Conv2d[conv]
%1211 : Float(1, 16, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1206, %213, %214, %215, %216), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1213 : Float(1, 16, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv4]/ReLU[activ]
%1219 : Float(1, 32, 55, 55) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1213, %218, %219), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv5]/Conv2d[conv]
%1224 : Float(1, 32, 55, 55) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1219, %220, %221, %222, %223), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1226 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/SqnxtConv[conv5]/ReLU[activ]
%1227 : Float(1, 32, 55, 55) = aten::add[alpha={1}](%1226, %1161), scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]
%1229 : Float(1, 32, 55, 55) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage1]/SqnxtUnit[unit6]/ReLU[activ]
%1235 : Float(1, 64, 28, 28) = aten::_convolution[stride=[2, 2], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1229, %260, %261), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/Conv2d[conv]
%1240 : Float(1, 64, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1235, %262, %263, %264, %265), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/BatchNorm2d[bn]
%1242 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/ReLU[activ]
%1244 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/ReLU[activ]
%1250 : Float(1, 32, 28, 28) = aten::_convolution[stride=[2, 2], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1229, %225, %226), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv1]/Conv2d[conv]
%1255 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1250, %227, %228, %229, %230), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1257 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv1]/ReLU[activ]
%1263 : Float(1, 16, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1257, %232, %233), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv2]/Conv2d[conv]
%1268 : Float(1, 16, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1263, %234, %235, %236, %237), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1270 : Float(1, 16, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv2]/ReLU[activ]
%1276 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1270, %239, %240), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv3]/Conv2d[conv]
%1281 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1276, %241, %242, %243, %244), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1283 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv3]/ReLU[activ]
%1289 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1283, %246, %247), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv4]/Conv2d[conv]
%1294 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1289, %248, %249, %250, %251), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1296 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv4]/ReLU[activ]
%1302 : Float(1, 64, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1296, %253, %254), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv5]/Conv2d[conv]
%1307 : Float(1, 64, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1302, %255, %256, %257, %258), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1309 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/SqnxtConv[conv5]/ReLU[activ]
%1310 : Float(1, 64, 28, 28) = aten::add[alpha={1}](%1309, %1244), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]
%1312 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit1]/ReLU[activ]
%1314 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/ReLU[activ]
%1320 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1314, %267, %268), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv1]/Conv2d[conv]
%1325 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1320, %269, %270, %271, %272), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1327 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv1]/ReLU[activ]
%1333 : Float(1, 16, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1327, %274, %275), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv2]/Conv2d[conv]
%1338 : Float(1, 16, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1333, %276, %277, %278, %279), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1340 : Float(1, 16, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv2]/ReLU[activ]
%1346 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1340, %281, %282), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv3]/Conv2d[conv]
%1351 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1346, %283, %284, %285, %286), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1353 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv3]/ReLU[activ]
%1359 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1353, %288, %289), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv4]/Conv2d[conv]
%1364 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1359, %290, %291, %292, %293), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1366 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv4]/ReLU[activ]
%1372 : Float(1, 64, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1366, %295, %296), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv5]/Conv2d[conv]
%1377 : Float(1, 64, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1372, %297, %298, %299, %300), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1379 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/SqnxtConv[conv5]/ReLU[activ]
%1380 : Float(1, 64, 28, 28) = aten::add[alpha={1}](%1379, %1314), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]
%1382 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit2]/ReLU[activ]
%1384 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/ReLU[activ]
%1390 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1384, %302, %303), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv1]/Conv2d[conv]
%1395 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1390, %304, %305, %306, %307), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1397 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv1]/ReLU[activ]
%1403 : Float(1, 16, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1397, %309, %310), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv2]/Conv2d[conv]
%1408 : Float(1, 16, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1403, %311, %312, %313, %314), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1410 : Float(1, 16, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv2]/ReLU[activ]
%1416 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1410, %316, %317), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv3]/Conv2d[conv]
%1421 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1416, %318, %319, %320, %321), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1423 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv3]/ReLU[activ]
%1429 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1423, %323, %324), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv4]/Conv2d[conv]
%1434 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1429, %325, %326, %327, %328), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1436 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv4]/ReLU[activ]
%1442 : Float(1, 64, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1436, %330, %331), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv5]/Conv2d[conv]
%1447 : Float(1, 64, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1442, %332, %333, %334, %335), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1449 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/SqnxtConv[conv5]/ReLU[activ]
%1450 : Float(1, 64, 28, 28) = aten::add[alpha={1}](%1449, %1384), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]
%1452 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit3]/ReLU[activ]
%1454 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/ReLU[activ]
%1460 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1454, %337, %338), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv1]/Conv2d[conv]
%1465 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1460, %339, %340, %341, %342), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1467 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv1]/ReLU[activ]
%1473 : Float(1, 16, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1467, %344, %345), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv2]/Conv2d[conv]
%1478 : Float(1, 16, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1473, %346, %347, %348, %349), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1480 : Float(1, 16, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv2]/ReLU[activ]
%1486 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1480, %351, %352), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv3]/Conv2d[conv]
%1491 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1486, %353, %354, %355, %356), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1493 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv3]/ReLU[activ]
%1499 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1493, %358, %359), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv4]/Conv2d[conv]
%1504 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1499, %360, %361, %362, %363), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1506 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv4]/ReLU[activ]
%1512 : Float(1, 64, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1506, %365, %366), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv5]/Conv2d[conv]
%1517 : Float(1, 64, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1512, %367, %368, %369, %370), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1519 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/SqnxtConv[conv5]/ReLU[activ]
%1520 : Float(1, 64, 28, 28) = aten::add[alpha={1}](%1519, %1454), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]
%1522 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit4]/ReLU[activ]
%1524 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/ReLU[activ]
%1530 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1524, %372, %373), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv1]/Conv2d[conv]
%1535 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1530, %374, %375, %376, %377), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1537 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv1]/ReLU[activ]
%1543 : Float(1, 16, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1537, %379, %380), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv2]/Conv2d[conv]
%1548 : Float(1, 16, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1543, %381, %382, %383, %384), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1550 : Float(1, 16, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv2]/ReLU[activ]
%1556 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1550, %386, %387), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv3]/Conv2d[conv]
%1561 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1556, %388, %389, %390, %391), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1563 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv3]/ReLU[activ]
%1569 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1563, %393, %394), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv4]/Conv2d[conv]
%1574 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1569, %395, %396, %397, %398), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1576 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv4]/ReLU[activ]
%1582 : Float(1, 64, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1576, %400, %401), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv5]/Conv2d[conv]
%1587 : Float(1, 64, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1582, %402, %403, %404, %405), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1589 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/SqnxtConv[conv5]/ReLU[activ]
%1590 : Float(1, 64, 28, 28) = aten::add[alpha={1}](%1589, %1524), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]
%1592 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit5]/ReLU[activ]
%1594 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/ReLU[activ]
%1600 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1594, %407, %408), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv1]/Conv2d[conv]
%1605 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1600, %409, %410, %411, %412), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1607 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv1]/ReLU[activ]
%1613 : Float(1, 16, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1607, %414, %415), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv2]/Conv2d[conv]
%1618 : Float(1, 16, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1613, %416, %417, %418, %419), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1620 : Float(1, 16, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv2]/ReLU[activ]
%1626 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1620, %421, %422), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv3]/Conv2d[conv]
%1631 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1626, %423, %424, %425, %426), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1633 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv3]/ReLU[activ]
%1639 : Float(1, 32, 28, 28) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1633, %428, %429), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv4]/Conv2d[conv]
%1644 : Float(1, 32, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1639, %430, %431, %432, %433), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1646 : Float(1, 32, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv4]/ReLU[activ]
%1652 : Float(1, 64, 28, 28) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1646, %435, %436), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv5]/Conv2d[conv]
%1657 : Float(1, 64, 28, 28) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1652, %437, %438, %439, %440), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1659 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/SqnxtConv[conv5]/ReLU[activ]
%1660 : Float(1, 64, 28, 28) = aten::add[alpha={1}](%1659, %1594), scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]
%1662 : Float(1, 64, 28, 28) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage2]/SqnxtUnit[unit6]/ReLU[activ]
%1668 : Float(1, 128, 14, 14) = aten::_convolution[stride=[2, 2], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1662, %477, %478), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/Conv2d[conv]
%1673 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1668, %479, %480, %481, %482), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/BatchNorm2d[bn]
%1675 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/ReLU[activ]
%1677 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/ReLU[activ]
%1683 : Float(1, 64, 14, 14) = aten::_convolution[stride=[2, 2], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1662, %442, %443), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv1]/Conv2d[conv]
%1688 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1683, %444, %445, %446, %447), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1690 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv1]/ReLU[activ]
%1696 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1690, %449, %450), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv2]/Conv2d[conv]
%1701 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1696, %451, %452, %453, %454), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1703 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv2]/ReLU[activ]
%1709 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1703, %456, %457), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv3]/Conv2d[conv]
%1714 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1709, %458, %459, %460, %461), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1716 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv3]/ReLU[activ]
%1722 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1716, %463, %464), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv4]/Conv2d[conv]
%1727 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1722, %465, %466, %467, %468), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1729 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv4]/ReLU[activ]
%1735 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1729, %470, %471), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv5]/Conv2d[conv]
%1740 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1735, %472, %473, %474, %475), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1742 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/SqnxtConv[conv5]/ReLU[activ]
%1743 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%1742, %1677), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]
%1745 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit1]/ReLU[activ]
%1747 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/ReLU[activ]
%1753 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1747, %484, %485), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv1]/Conv2d[conv]
%1758 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1753, %486, %487, %488, %489), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1760 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv1]/ReLU[activ]
%1766 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1760, %491, %492), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv2]/Conv2d[conv]
%1771 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1766, %493, %494, %495, %496), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1773 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv2]/ReLU[activ]
%1779 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1773, %498, %499), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv3]/Conv2d[conv]
%1784 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1779, %500, %501, %502, %503), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1786 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv3]/ReLU[activ]
%1792 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1786, %505, %506), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv4]/Conv2d[conv]
%1797 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1792, %507, %508, %509, %510), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1799 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv4]/ReLU[activ]
%1805 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1799, %512, %513), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv5]/Conv2d[conv]
%1810 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1805, %514, %515, %516, %517), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1812 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/SqnxtConv[conv5]/ReLU[activ]
%1813 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%1812, %1747), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]
%1815 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit2]/ReLU[activ]
%1817 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/ReLU[activ]
%1823 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1817, %519, %520), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv1]/Conv2d[conv]
%1828 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1823, %521, %522, %523, %524), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1830 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv1]/ReLU[activ]
%1836 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1830, %526, %527), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv2]/Conv2d[conv]
%1841 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1836, %528, %529, %530, %531), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1843 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv2]/ReLU[activ]
%1849 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1843, %533, %534), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv3]/Conv2d[conv]
%1854 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1849, %535, %536, %537, %538), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1856 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv3]/ReLU[activ]
%1862 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1856, %540, %541), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv4]/Conv2d[conv]
%1867 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1862, %542, %543, %544, %545), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1869 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv4]/ReLU[activ]
%1875 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1869, %547, %548), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv5]/Conv2d[conv]
%1880 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1875, %549, %550, %551, %552), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1882 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/SqnxtConv[conv5]/ReLU[activ]
%1883 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%1882, %1817), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]
%1885 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit3]/ReLU[activ]
%1887 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/ReLU[activ]
%1893 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1887, %554, %555), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv1]/Conv2d[conv]
%1898 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1893, %556, %557, %558, %559), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1900 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv1]/ReLU[activ]
%1906 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1900, %561, %562), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv2]/Conv2d[conv]
%1911 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1906, %563, %564, %565, %566), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1913 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv2]/ReLU[activ]
%1919 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1913, %568, %569), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv3]/Conv2d[conv]
%1924 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1919, %570, %571, %572, %573), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1926 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv3]/ReLU[activ]
%1932 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1926, %575, %576), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv4]/Conv2d[conv]
%1937 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1932, %577, %578, %579, %580), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv4]/BatchNorm2d[bn]
%1939 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv4]/ReLU[activ]
%1945 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1939, %582, %583), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv5]/Conv2d[conv]
%1950 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1945, %584, %585, %586, %587), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv5]/BatchNorm2d[bn]
%1952 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/SqnxtConv[conv5]/ReLU[activ]
%1953 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%1952, %1887), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]
%1955 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit4]/ReLU[activ]
%1957 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/ReLU[activ]
%1963 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1957, %589, %590), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv1]/Conv2d[conv]
%1968 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1963, %591, %592, %593, %594), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv1]/BatchNorm2d[bn]
%1970 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv1]/ReLU[activ]
%1976 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1970, %596, %597), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv2]/Conv2d[conv]
%1981 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1976, %598, %599, %600, %601), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv2]/BatchNorm2d[bn]
%1983 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv2]/ReLU[activ]
%1989 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1983, %603, %604), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv3]/Conv2d[conv]
%1994 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%1989, %605, %606, %607, %608), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv3]/BatchNorm2d[bn]
%1996 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv3]/ReLU[activ]
%2002 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%1996, %610, %611), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv4]/Conv2d[conv]
%2007 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2002, %612, %613, %614, %615), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv4]/BatchNorm2d[bn]
%2009 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv4]/ReLU[activ]
%2015 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2009, %617, %618), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv5]/Conv2d[conv]
%2020 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2015, %619, %620, %621, %622), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv5]/BatchNorm2d[bn]
%2022 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/SqnxtConv[conv5]/ReLU[activ]
%2023 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%2022, %1957), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]
%2025 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit5]/ReLU[activ]
%2027 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/ReLU[activ]
%2033 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2027, %624, %625), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv1]/Conv2d[conv]
%2038 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2033, %626, %627, %628, %629), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv1]/BatchNorm2d[bn]
%2040 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv1]/ReLU[activ]
%2046 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2040, %631, %632), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv2]/Conv2d[conv]
%2051 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2046, %633, %634, %635, %636), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv2]/BatchNorm2d[bn]
%2053 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv2]/ReLU[activ]
%2059 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2053, %638, %639), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv3]/Conv2d[conv]
%2064 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2059, %640, %641, %642, %643), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv3]/BatchNorm2d[bn]
%2066 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv3]/ReLU[activ]
%2072 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2066, %645, %646), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv4]/Conv2d[conv]
%2077 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2072, %647, %648, %649, %650), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv4]/BatchNorm2d[bn]
%2079 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv4]/ReLU[activ]
%2085 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2079, %652, %653), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv5]/Conv2d[conv]
%2090 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2085, %654, %655, %656, %657), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv5]/BatchNorm2d[bn]
%2092 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/SqnxtConv[conv5]/ReLU[activ]
%2093 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%2092, %2027), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]
%2095 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit6]/ReLU[activ]
%2097 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/ReLU[activ]
%2103 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2097, %659, %660), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv1]/Conv2d[conv]
%2108 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2103, %661, %662, %663, %664), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv1]/BatchNorm2d[bn]
%2110 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv1]/ReLU[activ]
%2116 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2110, %666, %667), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv2]/Conv2d[conv]
%2121 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2116, %668, %669, %670, %671), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv2]/BatchNorm2d[bn]
%2123 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv2]/ReLU[activ]
%2129 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2123, %673, %674), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv3]/Conv2d[conv]
%2134 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2129, %675, %676, %677, %678), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv3]/BatchNorm2d[bn]
%2136 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv3]/ReLU[activ]
%2142 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2136, %680, %681), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv4]/Conv2d[conv]
%2147 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2142, %682, %683, %684, %685), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv4]/BatchNorm2d[bn]
%2149 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv4]/ReLU[activ]
%2155 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2149, %687, %688), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv5]/Conv2d[conv]
%2160 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2155, %689, %690, %691, %692), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv5]/BatchNorm2d[bn]
%2162 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/SqnxtConv[conv5]/ReLU[activ]
%2163 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%2162, %2097), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]
%2165 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit7]/ReLU[activ]
%2167 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/ReLU[activ]
%2173 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2167, %694, %695), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv1]/Conv2d[conv]
%2178 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2173, %696, %697, %698, %699), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv1]/BatchNorm2d[bn]
%2180 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv1]/ReLU[activ]
%2186 : Float(1, 32, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2180, %701, %702), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv2]/Conv2d[conv]
%2191 : Float(1, 32, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2186, %703, %704, %705, %706), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv2]/BatchNorm2d[bn]
%2193 : Float(1, 32, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv2]/ReLU[activ]
%2199 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2193, %708, %709), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv3]/Conv2d[conv]
%2204 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2199, %710, %711, %712, %713), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv3]/BatchNorm2d[bn]
%2206 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv3]/ReLU[activ]
%2212 : Float(1, 64, 14, 14) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2206, %715, %716), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv4]/Conv2d[conv]
%2217 : Float(1, 64, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2212, %717, %718, %719, %720), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv4]/BatchNorm2d[bn]
%2219 : Float(1, 64, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv4]/ReLU[activ]
%2225 : Float(1, 128, 14, 14) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2219, %722, %723), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv5]/Conv2d[conv]
%2230 : Float(1, 128, 14, 14) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2225, %724, %725, %726, %727), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv5]/BatchNorm2d[bn]
%2232 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/SqnxtConv[conv5]/ReLU[activ]
%2233 : Float(1, 128, 14, 14) = aten::add[alpha={1}](%2232, %2167), scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]
%2235 : Float(1, 128, 14, 14) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage3]/SqnxtUnit[unit8]/ReLU[activ]
%2241 : Float(1, 256, 7, 7) = aten::_convolution[stride=[2, 2], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2235, %764, %765), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/Conv2d[conv]
%2246 : Float(1, 256, 7, 7) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2241, %766, %767, %768, %769), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/BatchNorm2d[bn]
%2248 : Float(1, 256, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[identity_conv]/ReLU[activ]
%2250 : Float(1, 256, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/ReLU[activ]
%2256 : Float(1, 128, 7, 7) = aten::_convolution[stride=[2, 2], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2235, %729, %730), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv1]/Conv2d[conv]
%2261 : Float(1, 128, 7, 7) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2256, %731, %732, %733, %734), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv1]/BatchNorm2d[bn]
%2263 : Float(1, 128, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv1]/ReLU[activ]
%2269 : Float(1, 64, 7, 7) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2263, %736, %737), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv2]/Conv2d[conv]
%2274 : Float(1, 64, 7, 7) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2269, %738, %739, %740, %741), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv2]/BatchNorm2d[bn]
%2276 : Float(1, 64, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv2]/ReLU[activ]
%2282 : Float(1, 128, 7, 7) = aten::_convolution[stride=[1, 1], padding=[0, 1], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2276, %743, %744), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv3]/Conv2d[conv]
%2287 : Float(1, 128, 7, 7) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2282, %745, %746, %747, %748), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv3]/BatchNorm2d[bn]
%2289 : Float(1, 128, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv3]/ReLU[activ]
%2295 : Float(1, 128, 7, 7) = aten::_convolution[stride=[1, 1], padding=[1, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2289, %750, %751), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv4]/Conv2d[conv]
%2300 : Float(1, 128, 7, 7) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2295, %752, %753, %754, %755), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv4]/BatchNorm2d[bn]
%2302 : Float(1, 128, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv4]/ReLU[activ]
%2308 : Float(1, 256, 7, 7) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2302, %757, %758), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv5]/Conv2d[conv]
%2313 : Float(1, 256, 7, 7) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2308, %759, %760, %761, %762), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv5]/BatchNorm2d[bn]
%2315 : Float(1, 256, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/SqnxtConv[conv5]/ReLU[activ]
%2316 : Float(1, 256, 7, 7) = aten::add[alpha={1}](%2315, %2250), scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]
%2318 : Float(1, 256, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/Sequential[stage4]/SqnxtUnit[unit1]/ReLU[activ]
%2324 : Float(1, 128, 7, 7) = aten::_convolution[stride=[1, 1], padding=[0, 0], dilation=[1, 1], transposed=0, output_padding=[0, 0], groups=1, benchmark=0, deterministic=0, cudnn_enabled=1](%2318, %771, %772), scope: SqueezeNext/Sequential[features]/SqnxtConv[final_block]/Conv2d[conv]
%2329 : Float(1, 128, 7, 7) = aten::batch_norm[training=0, momentum=0, eps=1e-05, cudnn_enabled=1](%2324, %773, %774, %775, %776), scope: SqueezeNext/Sequential[features]/SqnxtConv[final_block]/BatchNorm2d[bn]
%2331 : Float(1, 128, 7, 7) = aten::thresholdthreshold={0}, value={0}, scope: SqueezeNext/Sequential[features]/SqnxtConv[final_block]/ReLU[activ]
%2333 : Float(1, 128, 1, 1) = aten::avg_pool2dkernel_size=[7, 7], stride=[1, 1], padding=[0, 0], ceil_mode=0, count_include_pad=1, scope: SqueezeNext/Sequential[features]/AvgPool2d[final_pool]
%2334 : Long() = aten::sizedim=0, scope: SqueezeNext
%2335 : Long() = prim::Constantvalue={-1}, scope: SqueezeNext
%2336 : Dynamic = aten::stack[dim=0](%2334, %2335), scope: SqueezeNext
%2337 : Float(1, 128) = aten::view(%2333, %2336), scope: SqueezeNext
%2338 : Float(128!, 1000!) = aten::t(%778), scope: SqueezeNext/Linear[output]
%2339 : Float(1, 1000) = aten::expandsize=[1, 1000], implicit=1, scope: SqueezeNext/Linear[output]
%2340 : Float(1, 1000) = aten::addmm[beta={1}, alpha={1}](%2339, %2337, %2338), scope: SqueezeNext/Linear[output]
return (%2340);
}
, False

Error: Failing in Transpose layer (Cannot permute batch dimension. Result may be wrong )

graph node: CRNN type: onnx::Transpose inputs: ['68'] outputs: ['CRNN'] name in state_dict: attrs: {'perm': [2, 0, 1]} is_terminal: False Converting transpose ... !!! Cannot permute batch dimension. Result may be wrong !!! Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/ben/.local/lib/python3.6/site-packages/pytorch2keras/converter.py", line 143, in pytorch_to_keras if node_inputs[0] in model_inputs: IndexError: list index out of range

Porting inception based architectures (inception v4)

Hi, once again thanks for awesome work.
It really helps with shortening pytorch to production path (pytorch=>keras=>tf).

I was running my pytorch model based on inception4 architecture.

I encountered this error during the run:

ValueError: Unsuported padding size for convolution

I guess this is due to my architecture being an inception4 architecture - it has non-symmetric filters and this is solved differently in keras and pytorch. In keras it is solved via 'same' convolutions, in pytorch - via different paddings. Investigating deeper - I indeed found out that conv layers with assymmetric padding in pytorch are the culprits.

Just compare these links:

I overcame this by essentially a hack, but maybe you will give some advice / commit on how to do it properly?

In a nutshell I did this.
I can do a PR with this change, if you want.

        if node.padding[0] != node.padding[1]:
            # originally this line was not commented
            # raise ValueError('Unsuported padding size for convolution')
            
            # quick fix for inception architectures
            # refer here for more info https://github.com/fchollet/keras/blob/master/keras/applications/inception_v3.py
            border_mode = 'same'
        else:
            # this code initially was under no condition
            padding = node.padding[0]
            if padding > 0:
                padding_name = output_name + '_pad'
                padding_layer = keras.layers.ZeroPadding2D(
                    padding=node.padding,
                    name=padding_name
                )
                layers[padding_name] = padding_layer(layers[input_name])
                input_name = padding_name      
                
            # this line below also was applied unconditionally
            border_mode = 'valid'

Anyway - which proper solution would you suggest for this edge case?

pytorch 0.3

After upgrade pytorch to version 0.3, the test file alexnet.py can not work correctly. How to solve this problem?

about the branch about supporting pytorch0.3.0 or 0.3.1

after reading all the issue, there was the branch which could support the pytorch0.3.0,
would you mind sharing this branch again.
Thanks
@nerox8664

NEW ErrorUnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1124: ordinal not in range(128)

Using TensorFlow backend.
Traceback (most recent call last):
File "pytorch_fpn-res2keras-tf.py", line 18, in
pytorch_model = torch.load('./_0620_80000.pth')
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/serialization.py", line 303, in load
return _load(f, map_location, pickle_module)
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/serialization.py", line 469, in _load
result = unpickler.load()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1124: ordinal not in range(128)
@nerox8664 would u mind sharing all of your environment? I mean, for example, numpy'version included. The repo work well about a mouth ago, but now, it doesn't work. And, I have upgraded something for the past a mouth.
Thanks.
looking forward to any replies.

KeyError: u'prim::PythonOp'

When I convert the customized network from Pytorch to Keras-tensorflow, the error is following:
Traceback (most recent call last):
File "pytorch_fpn-res2keras-tf.py", line 26, in
keras_model = pytorch_to_keras(pytorch_model, input_var, (3, 384, 384), verbose=True)
File "/data/xxxx/framework_exchange/pytorch2keras/pytorch2keras/converter.py", line 160, in pytorch_to_keras
AVAILABLE_CONVERTERS[node_type](
KeyError: u'prim::PythonOp'

@nerox8664 @justasbr
Looking forward to any replies. Thanks

problem with loading the keras model

Hi,

I followed the instruction to get keras model from a torch model. It is (probably) successful, and I save it and everything till here is just fine.
However, when I try to load again the saved keras model, I get the following error:

Using TensorFlow backend.
2018-12-13 11:49:04.256785: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Traceback (most recent call last):
File "D:/Github packages/deep-head-pose-master-python3/deep-head-pose-master/core2/testing.py", line 4, in <module>
model = keras.models.load_model('keras_model.h5')
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\engine\saving.py", line 419, in load_model
model = _deserialize_model(f, custom_objects, compile)
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\engine\saving.py", line 225, in _deserialize_model
model = model_from_config(model_config, custom_objects=custom_objects)
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\engine\saving.py", line 458, in model_from_config return deserialize(config, custom_objects=custom_objects)
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\layers\__init__.py", line 55, in deserialize printable_module_name='layer')
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\utils\generic_utils.py", line 145, in deserialize_keras_object list(custom_objects.items())))
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\engine\network.py", line 1032, in from_config process_node(layer, node_data)
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\engine\network.py", line 991, in process_node layer(unpack_singleton(input_tensors), **kwargs)
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\engine\base_layer.py", line 457, in __call__ output = self.call(inputs, **kwargs)
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\keras\layers\core.py", line 687, in call return self.function(inputs, **arguments)
File "C:\Users\cnq\AppData\Local\Continuum\miniconda3\envs\torch_new\lib\site-packages\pytorch2keras-0.1.12-py3.6.egg\pytorch2keras\layers.py", line 1330, in target_layer def target_layer(x):
NameError: name 'keras' is not defined

Any idea?

OS: Win 7
Python 3.6
Version [e.g. v0.1.11]

Error in convert_constant

It seems that there is an error in convert.convert_constant function, where it is defined as

def convert_constant(params, w_name, scope_name, inputs, layers, weights):
    def target_layer(params=params):
        return keras.backend.constant(np.float32(params['value']))

    lambda_layer = keras.layers.Lambda(target_layer)
    layers[scope_name] = lambda_layer(layers[inputs[0]])

while I believe the layers[inputs[0]] argument to lambda_layer is supposed to be a dummy argument to bypass compatibility check in keras base_layer and the params variable from the convert_constant scope should be the real argument and contains the actual constant values.

Therefore, the correct way to define target_layer should be:

    def target_layer(x, params=params):
        return keras.backend.constant(np.float32(params['value']))

instead.

From channels_first to channels_last?

Is it possible to convert model from channels_first to channels_last(default Tensorflow tensor dim ordering)?

KeyError: 'Constant' while converting simple feedforward model

Hi!
I have a problem in converting simple feedforward net
Here is my code:

from torch import nn, zeros, LongTensor, FloatTensor, onnx
from torch.autograd import Variable
import torch.nn.functional as F
from converter import pytorch_to_keras

class SimpleNet(nn.Module):
    def __init__(self, input_size):
        super(SimpleNet, self).__init__()
        
        self.embedd = nn.Embedding(input_size, 100)
        self.fc1 = nn.Linear(100, 200)
        self.fc2 = nn.Linear(200, 200)
        self.fc3 = nn.Linear(200, 300)
                
    def forward(self, input):
        r_0 = input.mm(self.embedd.weight)
        r_1 = F.tanh(self.fc1(r_0).div(10) )
        r_2 = F.tanh(self.fc2(r_1).div(10) )  + r_1
        r_3 = F.tanh(self.fc3(r_2).div(10) )
        return r_3

# prepare fake input
indices = [1,2,3,4,5]
input = zeros(1000).type(FloatTensor)

for i in indices:
    input[i] = 1

input = Variable(input)
input = input.view(1, -1)

# create forward pass with fake input
simple_net = SimpleNet(1000)
out = simple_net(input)

k_model = pytorch_to_keras(simple_net, input, (1, 1000), verbose=True)

This gives me this output with error:

graph(%0 : Float(1, 1000)
      %1 : Float(1000, 100)
      %2 : Float(200, 100)
      %3 : Float(200)
      %4 : Float(200, 200)
      %5 : Float(200)
      %6 : Float(300, 200)
      %7 : Float(300)) {
  %8 : UNKNOWN_TYPE = Constant[value={0}](), scope: SimpleNet
  %9 : Float(1, 100) = Gemm[alpha=1, beta=0, broadcast=1](%0, %1, %8), scope: SimpleNet
  %10 : Float(1, 200) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%9, %2, %3), scope: SimpleNet/Linear[fc1]
  %11 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
  %12 : Float(1, 200) = Div[broadcast=1](%10, %11), scope: SimpleNet
  %13 : Float(1, 200) = Tanh(%12), scope: SimpleNet
  %14 : Float(1, 200) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%13, %4, %5), scope: SimpleNet/Linear[fc2]
  %15 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
  %16 : Float(1, 200) = Div[broadcast=1](%14, %15), scope: SimpleNet
  %17 : Float(1, 200) = Tanh(%16), scope: SimpleNet
  %18 : Float(1, 200) = Add(%17, %13), scope: SimpleNet
  %19 : Float(1, 300) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%18, %6, %7), scope: SimpleNet/Linear[fc3]
  %20 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
  %21 : Float(1, 300) = Div[broadcast=1](%19, %20), scope: SimpleNet
  %22 : Float(1, 300) = Tanh(%21), scope: SimpleNet
  return (%22);
}

[22 defined in (%22 : Float(1, 300) = Tanh(%21), scope: SimpleNet
)]
['embedd.weight', 'fc1.weight', 'fc1.bias', 'fc2.weight', 'fc2.bias', 'fc3.weight', 'fc3.bias']
 ____ 
graph node: SimpleNet
type: Constant
inputs: ['input']
outputs: ['SimpleNet']
name in state_dict: 
attrs: {'value': 
 0
[torch.FloatTensor of size 1]
}
is_terminal: False
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-34-ee7856e5a4c6> in <module>()
----> 1 k_model = pytorch_to_keras(simple_net, input, (1, 1000), verbose=True)  #we should specify shape of the input tensor

~/workspace/pytorch2keras/pytorch2keras/converter.py in pytorch_to_keras(model, args, input_shape, change_ordering, training, verbose)
    154             print('attrs:', node_attrs)
    155             print('is_terminal:', node_id in graph_outputs)
--> 156         AVAILABLE_CONVERTERS[node_type](
    157             node_attrs,
    158             node_weights_name, node_id,

KeyError: 'Constant'

I should note, that i succesfully convert this model to onnx format:

torch_out = onnx._export(simple_net,
                         input,
                         "test_simple_net.onnx",
                         export_params=True,
                         verbose=True
                         )

graph(%0 : Float(1, 1000)
      %1 : Float(1000, 100)
      %2 : Float(200, 100)
      %3 : Float(200)
      %4 : Float(200, 200)
      %5 : Float(200)
      %6 : Float(300, 200)
      %7 : Float(300)) {
  %8 : UNKNOWN_TYPE = Constant[value={0}](), scope: SimpleNet
  %9 : Float(1, 100) = Gemm[alpha=1, beta=0, broadcast=1](%0, %1, %8), scope: SimpleNet
  %10 : Float(1, 200) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%9, %2, %3), scope: SimpleNet/Linear[fc1]
  %11 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
  %12 : Float(1, 200) = Div[broadcast=1](%10, %11), scope: SimpleNet
  %13 : Float(1, 200) = Tanh(%12), scope: SimpleNet
  %14 : Float(1, 200) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%13, %4, %5), scope: SimpleNet/Linear[fc2]
  %15 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
  %16 : Float(1, 200) = Div[broadcast=1](%14, %15), scope: SimpleNet
  %17 : Float(1, 200) = Tanh(%16), scope: SimpleNet
  %18 : Float(1, 200) = Add(%17, %13), scope: SimpleNet
  %19 : Float(1, 300) = Gemm[alpha=1, beta=1, broadcast=1, transB=1](%18, %6, %7), scope: SimpleNet/Linear[fc3]
  %20 : UNKNOWN_TYPE = Constant[value={10}](), scope: SimpleNet
  %21 : Float(1, 300) = Div[broadcast=1](%19, %20), scope: SimpleNet
  %22 : Float(1, 300) = Tanh(%21), scope: SimpleNet
  return (%22);
}

Could you add "Constant" to AVAILABLE_CONVERTERS ?
thank you

KeyError: 'module.weight'

Describe the bug
I am trying to convert a simple Pytorch Gender Classifier to Keras (model is provided below). Whenever I run model = pytorch_to_keras(model, orig_img, [np.shape(orig_img.cpu().numpy())], names='short'), I get

Traceback (most recent call last):
File "main.py", line 154, in
main(args)
File "main.py", line 92, in main
model = pytorch_to_keras(model, orig_img, [np.shape(orig_img.cpu().numpy())], names='short')
File "/mnt/c/Users/Yannis/CIFAR10S/real_world_experiments/CEM/venv/lib/python3.6/site-packages/pytorch2keras/converter.py", line 325, in pytorch_to_keras
names
File "/mnt/c/Users/Yannis/CIFAR10S/real_world_experiments/CEM/venv/lib/python3.6/site-packages/pytorch2keras/convolution_layers.py", line 35, in convert_conv
if len(weights[weights_name].numpy().shape) == 5: # 3D conv
KeyError: 'module.weight'

To Reproduce
Here is my pytorch model:

class resnet_modified_small(nn.Module):
    def base_size(self): return 512
    def rep_size(self): return 1024

    def __init__(self, n_classes):
        super(resnet_modified_small, self).__init__()
        self.resnet = tv.models.resnet34(pretrained=True)

        # define layers
        self.n_classes = n_classes
        self.linear = nn.Linear(7 * 7 * self.base_size(), self.rep_size())
        self.cls = nn.Linear(self.rep_size(), self.n_classes)
        self.dropout2d = nn.Dropout2d(.5)
        self.dropout = nn.Dropout(.5)
        self.relu = nn.LeakyReLU()
        initLinear(self.linear)

    def forward(self, out0):
        x = self.resnet.conv1(out0)
        x = self.resnet.bn1(x)
        x = self.resnet.relu(x)
        out1 = self.resnet.maxpool(x)

        out2 = self.resnet.layer1(out1)
        out3 = self.resnet.layer2(out2)
        out4 = self.resnet.layer3(out3)
        out5 = self.resnet.layer4(out4)
     
        x = self.dropout2d(out5)

        features = self.dropout(self.relu(self.linear(x.view(-1, 7*7*self.base_size()))))
        cls_scores = self.cls(features)
        return [out0, out1, out2, out3, out4, out5, features, cls_scores]

Environment:

OS: Bash for Windows
Python 3.6
Pytorch 0.4.1

ModuleNotFoundError: No module named 'converter'

Feature request
A clear and concise description of what the problem is. Ex. My layer hasn't supported yet [...]

Additional context
Add any other context or screenshots about the feature request here.
I follow the installation steps but get the error , when from converter import pytorch_to_keras
Is it not suitable for python3 ?

Error on running test file to convert resnet50 to keras

Getting this error:
KeyError: <tf.Tensor 'maxpool0.13212125626892746/transpose_1:0' shape=(?, 64, 56, 56) dtype=float32>

Add support of the PyTorch 0.3

There are some problems with new PyTorch release:

The current traverse function is no longer accurate.
Number of layers was renamed.
Difference in the arguments / calculation.

What to do:

MaxPooling / AveragePooling issue

With some stride / padding / kernel size combination results may be different.

For example:

stride: 2
padding: 1
kernel_size: 3

Does conv2D transpose converting work properly?

Should we simply add zeropadding before convolution transpose or we should use different advanced feature? :)
Output shapes in pytorch and keras are different

Error:about the new customized layer

@nerox8664 Hi, we train a model with customized, called SegmentConsensus, layer by torch.0.4.0, and getting the following error:
Using TensorFlow backend.
Freezing BatchNorm2D except the first one.
Freezing BatchNorm2D except the first one.
Traceback (most recent call last):
File "pytorch_trn-res2keras-tf.py", line 27, in
keras_model = pytorch_to_keras(pytorch_model, input_var, (4, 3, 224, 224), verbose=True)
File "/home/intellif/framework_exchange/pytorch2keras/pytorch2keras/converter.py", line 91, in pytorch_to_keras
trace, torch_out = torch.jit.get_trace_graph(model, tuple(args))
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/contextlib.py", line 77, in exit
self.gen.throw(type, value, traceback)
File "/home/intellif/framework_exchange/pytorch2keras/pytorch2keras/converter.py", line 29, in set_training
yield
File "/home/intellif/framework_exchange/pytorch2keras/pytorch2keras/converter.py", line 91, in pytorch_to_keras
trace, torch_out = torch.jit.get_trace_graph(model, tuple(args))
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/jit/init.py", line 255, in get_trace_graph
return LegacyTracedModule(f, nderivs=nderivs)(*args, **kwargs)
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/jit/init.py", line 288, in forward
out = self.inner(*trace_inputs)
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call
result = self._slow_forward(*input, **kwargs)
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/nn/modules/module.py", line 479, in _slow_forward
result = self.forward(*input, **kwargs)
File "/home/intellif/framework_exchange/pytorch2keras/models.py", line 338, in forward
output = self.consensus(base_out)
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/nn/modules/module.py", line 489, in call
result = self._slow_forward(*input, **kwargs)
File "/home/intellif/anaconda2/envs/py3torch4/lib/python3.5/site-packages/torch/nn/modules/module.py", line 479, in _slow_forward
result = self.forward(*input, **kwargs)
File "/home/intellif/framework_exchange/pytorch2keras/ops/basic_ops.py", line 47, in forward
return SegmentConsensus(self.consensus_type, self.dim)(input)
RuntimeError: Attempted to trace SegmentConsensus, but tracing of legacy functions is not supported

How to understand tracing of legacy functions is not supported

question

please，whether batchnorm1d is supported？？

Close bracket

Close the bracket at https://github.com/nerox8664/pytorch2keras/blob/master/pytorch2keras/layers.py#L23

Unable to convert resnet18 model

While running resnet18.py in tests/ I encountered the

KeyError: 'Layer is not currently supported.'

So is there any other way to convert resnet18 model from torch to keras?

ImportError: cannot import name '_unique_state_dict'

It seems like 49dc427 breaks 0.2 compatibility

ImportError                               Traceback (most recent call last)
<ipython-input-9-9e4002883d78> in <module>()
      1 import sys
      2 sys.path.insert(0, '../pytorch2keras/pytorch2keras')
----> 3 from converter import pytorch_to_keras
      4 k_model = pytorch_to_keras(model, x, (3, 224, 224), verbose=True)
      5 #we should specify shape of the input tensor

~/projects/pytorch2keras/pytorch2keras/converter.py in <module>()
      8 import torch.serialization
      9 import contextlib
---> 10 from torch.jit import _unique_state_dict
     11 
     12 from layers import AVAILABLE_CONVERTERS

ImportError: cannot import name '_unique_state_dict'

_unique_state_dict was added in 0.4 version: pytorch/pytorch@e6cbe84

Dense layer conversion bug with multiple inputs

Describe the bug
Multiple linear layers with multiple inputs do not work. When the input shapes are identical, it works (e.g both inputs expect 5 units or both expect 10 units, but when one is 5 and the other is 10 it does not work).

To Reproduce

class LayerTest(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(10, 10)
        self.fc2 = nn.Linear(5, 10)

    def forward(self, x, y):
        x = self.fc1(x)
        y = self.fc2(y)
        return x + y

model = LayerTest().eval()
input_x = torch.FloatTensor(np.random.uniform(0, 1, (1, 10)))
input_y = torch.FloatTensor(np.random.uniform(0, 1, (1, 5)))
output = model(input_x, input_y)
k_model = pytorch_to_keras(model, [input_x, input_y], [(10,), (5,)], verbose=True)

Expected behavior
This should convert properly.

Logs

Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/henry/code/altum/model-trainer/export/export.py", line 54, in <module>
    k_model = pytorch_to_keras(model, [input_x, input_y], [(10,), (5,)], verbose=True)
  File "/usr/local/lib/python3.6/site-packages/pytorch2keras/converter.py", line 332, in pytorch_to_keras
    names
  File "/usr/local/lib/python3.6/site-packages/pytorch2keras/linear_layers.py", line 49, in convert_gemm
    layers[scope_name] = dense(layers[inputs[0]])
  File "/usr/local/lib/python3.6/site-packages/keras/engine/base_layer.py", line 436, in __call__
    self.set_weights(self._initial_weights)
  File "/usr/local/lib/python3.6/site-packages/keras/engine/base_layer.py", line 1057, in set_weights
    'provided weight shape ' + str(w.shape))
ValueError: Layer weight shape (5, 10) not compatible with provided weight shape (10, 10)

Share weights file？

Hi
Have there anybody successfully transferred the weights file to Keras?
Could you please share ResNet18 keras weights?
Thx!

Error when importing torch._C.ListType

I am using Pytorch v0.4.1.post2. When executing my code, I get an ImportError saying that torch._C.ListType does not exist.

The error happens in converter.py. Basically the conditions used if version.parse('0.4.1') < version.parse(torch.__version__) should only be true if the user is using Pytorch>=1.0.0 but are also true for versions v0.4.1.x.

I managed to work around this by replacing if version.parse('0.4.1') < version.parse(torch.__version__) with if version.parse('1.0.0') <= version.parse(torch.__version__). I can PR if you want.

can't pickle SwigPyObject objects

Thank you very much for your code.
I was able to convert my pytorch model to keras successfully. The summary of model was also correct. But when I wanted to save the keras model, I received the below error. I searched but didn't find anything useful. I appreciate any advice or suggestions.

TypeError: can't pickle SwigPyObject objects

Thanks,
Noosh

All layer names has been changed

During transfer, all layer names are changed, so it is confused to use because it is hard to find the layer name match the standard resnet101 name

ERROR!!!! When I run the converted model , it always got terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc

@nerox8664 Hi guys, after successfully getting the converted model, I got the error as following:
Using TensorFlow backend.
2018-06-28 11:24:45.607236: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-06-28 11:24:46.300184: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.62
pciBusID: 0000:88:00.0
totalMemory: 10.91GiB freeMemory: 10.75GiB
2018-06-28 11:24:46.300260: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-06-28 11:24:46.914014: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-06-28 11:24:46.914079: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929] 0
2018-06-28 11:24:46.914088: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0: N
2018-06-28 11:24:46.914566: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10390 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:88:00.0, compute capability: 6.1)
/home/wupeng/workshop/fashion-keypoint/venv-kp/local/lib/python2.7/site-packages/keras/models.py:252: UserWarning: No training configuration found in save file: the model was not compiled. Compile it manually.
warnings.warn('No training configuration found in save file: '
('image_path :', './data/train_0427/Images/dress/ca8b38cc7fb0ce9e35563766d12090b7.jpg')
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
Aborted (core dumped)

Input to InstanceNorm does not take Tensor

Trying to convert an Instance Normalization layer but never successfully done so. Did a test script as the following:

package version:
pytorch2keras: 0.1.10
tensorflow: 1.12
pytorch: 0.4.1
numpy: 1.15.1

# pytorch2tf_instancenorm_test.py

import numpy as np
import torch
from torch.autograd import Variable
from pytorch2keras import converter

class Pytorch2KerasTestNet(torch.nn.Module):
    def __init__(self):
        super(Pytorch2KerasTestNet, self).__init__()
        self.conv1 = ConvLayer(3, 32, kernel_size=9, stride=1)
        self.in1 = torch.nn.InstanceNorm2d(32, affine=True)
        self.relu = torch.nn.ReLU()

    def forward(self, x):
        y = self.relu(self.in1(self.conv1(x)))
        return y


class ConvLayer(torch.nn.Module):
    def __init__(self, in_channels, out_channels, kernel_size, stride):
        super(ConvLayer, self).__init__()
        reflection_padding = kernel_size // 2
        self.reflection_pad = torch.nn.ReflectionPad2d(reflection_padding)
        self.conv2d = torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride)

    def forward(self, x):
        out = self.reflection_pad(x)
        
        print("conv2d")
        out = self.conv2d(out)
        return out
        
model   = Pytorch2KerasTestNet()

input_np = np.random.uniform(0, 1, (1, 3, 224, 224))
input_var = Variable(torch.FloatTensor(input_np))
k_model = converter.pytorch_to_keras(model, input_var, [(3, 224, 224,)], verbose=True)

When running it using python pytorch2tf_instancenorm_test.py, it results in the following error:

2018-12-05 20:57:56.505141: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Using TensorFlow backend.
conv2d
graph(%0 : Float(1, 3, 224, 224)
      %1 : Float(32, 3, 9, 9)
      %2 : Float(32)
      %3 : Float(32)
      %4 : Float(32)) {
  %5 : Float(1, 3, 232, 232) = onnx::Pad[mode=reflect, pads=[0, 0, 4, 4, 0, 0, 4, 4]](%0), scope: Pytorch2KerasTestNet/ConvLayer[conv1]/ReflectionPad2d[reflection_pad]
  %6 : Float(1, 32, 224, 224) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[9, 9], pads=[0, 0, 0, 0], strides=[1, 1]](%5, %1, %2), scope: Pytorch2KerasTestNet/ConvLayer[conv1]/Conv2d[conv2d]
  %7 : Dynamic = onnx::Constant[value=<Tensor>](), scope: Pytorch2KerasTestNet/InstanceNorm2d[in1]
  %8 : Dynamic = onnx::Constant[value=<Tensor>](), scope: Pytorch2KerasTestNet/InstanceNorm2d[in1]
  %9 : Float(1, 32, 224, 224) = onnx::InstanceNormalization[epsilon=1e-05](%6, %7, %8), scope: Pytorch2KerasTestNet/InstanceNorm2d[in1]
  %10 : Float(1, 32, 224, 224) = onnx::Relu(%9), scope: Pytorch2KerasTestNet/ReLU[relu]
  return (%10);
}

[10 defined in (%10 : Float(1, 32, 224, 224) = onnx::Relu(%9), scope: Pytorch2KerasTestNet/ReLU[relu]
)]
[%5 : Float(1, 3, 232, 232) = onnx::Pad[mode=reflect, pads=[0, 0, 4, 4, 0, 0, 4, 4]](%0), scope: Pytorch2KerasTestNet/ConvLayer[conv1]/ReflectionPad2d[reflection_pad]
, %6 : Float(1, 32, 224, 224) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[9, 9], pads=[0, 0, 0, 0], strides=[1, 1]](%5, %1, %2), scope: Pytorch2KerasTestNet/ConvLayer[conv1]/Conv2d[conv2d]
, %7 : Dynamic = onnx::Constant[value=<Tensor>](), scope: Pytorch2KerasTestNet/InstanceNorm2d[in1]
, %8 : Dynamic = onnx::Constant[value=<Tensor>](), scope: Pytorch2KerasTestNet/InstanceNorm2d[in1]
, %9 : Float(1, 32, 224, 224) = onnx::InstanceNormalization[epsilon=1e-05](%6, %7, %8), scope: Pytorch2KerasTestNet/InstanceNorm2d[in1]
, %10 : Float(1, 32, 224, 224) = onnx::Relu(%9), scope: Pytorch2KerasTestNet/ReLU[relu]
]
Graph outputs: ['10']
State dict: ['conv1.conv2d.weight', 'conv1.conv2d.bias', 'in1.weight', 'in1.bias']
 ____ 
graph node: Pytorch2KerasTestNet/ConvLayer[conv1]/ReflectionPad2d[reflection_pad]
id: 5
type: onnx::Pad
inputs: ['input0']
outputs: ['Pytorch2KerasTestNet/ConvLayer[conv1]/ReflectionPad2d[reflection_pad]']
name in state_dict: conv1.reflection_pad
attrs: {'mode': 'reflect', 'pads': [0, 0, 4, 4, 0, 0, 4, 4]}
is_terminal: False
Converting padding...
 ____ 
graph node: Pytorch2KerasTestNet/ConvLayer[conv1]/Conv2d[conv2d]
id: 6
type: onnx::Conv
inputs: ['5']
outputs: ['Pytorch2KerasTestNet/ConvLayer[conv1]/Conv2d[conv2d]']
name in state_dict: conv1.conv2d
attrs: {'dilations': [1, 1], 'group': 1, 'kernel_shape': [9, 9], 'pads': [0, 0, 0, 0], 'strides': [1, 1]}
is_terminal: False
Converting convolution ...
 ____ 
graph node: Pytorch2KerasTestNet/InstanceNorm2d[in1]
id: 7
type: onnx::Constant
inputs: []
outputs: ['Pytorch2KerasTestNet/InstanceNorm2d[in1]']
name in state_dict: in1
attrs: {'value': tensor([0.8979, 0.3156, 0.1349, 0.2346, 0.8874, 0.7174, 0.3563, 0.0794, 0.9949,
        0.6409, 0.7893, 0.0516, 0.6400, 0.9812, 0.6296, 0.0555, 0.8294, 0.1040,
        0.7664, 0.8627, 0.5198, 0.5813, 0.6676, 0.4292, 0.9838, 0.8877, 0.0068,
        0.0730, 0.9636, 0.5794, 0.4843, 0.7476])}
is_terminal: False
Converting constant ...
 ____ 
graph node: Pytorch2KerasTestNet/InstanceNorm2d[in1]
id: 8
type: onnx::Constant
inputs: []
outputs: ['Pytorch2KerasTestNet/InstanceNorm2d[in1]']
name in state_dict: in1
attrs: {'value': tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}
is_terminal: False
Converting constant ...
 ____ 
graph node: Pytorch2KerasTestNet/InstanceNorm2d[in1]
id: 9
type: onnx::InstanceNormalization
inputs: ['6', '7', '8']
outputs: ['Pytorch2KerasTestNet/InstanceNorm2d[in1]']
name in state_dict: in1
attrs: {'epsilon': 1e-05}
is_terminal: False
Converting instancenorm ...
Traceback (most recent call last):
  File "pytorch2tf_instancenorm_test.py", line 45, in <module>
    k_model = converter.pytorch_to_keras(model, input_var, [(3, 224, 224,)], verbose=True)  
  File "/home/myang/py3env/local/lib/python3.6/site-packages/pytorch2keras/converter.py", line 316, in pytorch_to_keras
    names
  File "/home/myang/py3env/local/lib/python3.6/site-packages/pytorch2keras/layers.py", line 653, in convert_instancenorm
    layers[scope_name] = lambda_layer(layers[inputs[0]])
  File "/home/myang/py3env/local/lib/python3.6/site-packages/keras/engine/base_layer.py", line 457, in __call__
    output = self.call(inputs, **kwargs)
  File "/home/myang/py3env/local/lib/python3.6/site-packages/keras/layers/core.py", line 687, in call
    return self.function(inputs, **arguments)
  File "/home/myang/py3env/local/lib/python3.6/site-packages/pytorch2keras/layers.py", line 646, in target_layer
    param_initializers={'beta': tf.constant_initializer(beta), 'gamma': tf.constant_initializer(gamma)},
  File "/home/myang/py3env/local/lib/python3.6/site-packages/tensorflow/python/ops/init_ops.py", line 208, in __init__
    "tuple of values, or numpy.ndarray)." % type(value))
TypeError: Invalid type for initial value: <class 'tensorflow.python.framework.ops.Tensor'> (expected Python scalar, list or tuple of values, or numpy.ndarray).

The code is an excerpt from PyTorch Fast Style Transfer. Is this related to #37? Really trying to get it to work...

question

File "/usr/local/lib/python3.5/dist-packages/pytorch2keras/reshape_layers.py", line 122, in target_layer
return tf.squeeze(x, axis=axis)
NameError: name 'tf' is not defined
why happen to it，please！

model.save error

I convert the model from pytroch to keras successfully, but I can't save it, which report the following error

k_model = pytorch_to_keras(trained_model, input_var, [(3, 32, 32,)],change_ordering=True, verbose=False)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/site-packages/pytorch2keras/converter.py", line 341, in pytorch_to_keras
    conf = model.get_config()
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/site-packages/keras/engine/network.py", line 931, in get_config
    return copy.deepcopy(config)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 215, in _deepcopy_list
    append(deepcopy(a, memo))
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 220, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 220, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 220, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 220, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/copy.py", line 169, in deepcopy
    rv = reductor(4)
TypeError: can't pickle _thread.RLock objects

Problem with converting BatchNorm layer

I've added batchnorm layer to your dummy example and it crushed

the code:

from keras import backend as K
K.set_image_data_format('channels_first')
from pytorch2keras.converter import pytorch_to_keras
import torch 
import torch.nn as nn
import numpy as np
from torch.autograd import Variable

class TestConv2d(nn.Module):
    def __init__(self, inp=10, out=16, kernel_size=3):
        super(TestConv2d, self).__init__()
        self.conv2d = nn.Conv2d(inp, out, stride=1, kernel_size=kernel_size, bias=True)
        self.bn_1   = nn.BatchNorm2d(num_features = out)
    def forward(self, x):
        x = self.conv2d(x)
        x = self.bn_1(x)
        return x

model = TestConv2d()
input_np = np.random.uniform(0, 1, (1, 10, 32, 32))
input_var = Variable(torch.FloatTensor(input_np))

# we should specify shape of the input tensor
k_model = pytorch_to_keras(model, input_var, [(10, 32, 32,)], verbose=True)

the error:

graph(%0 : Float(1, 10, 32, 32)
      %1 : Float(16, 10, 3, 3)
      %2 : Float(16)
      %3 : Float(16)
      %4 : Float(16)
      %5 : Float(16)
      %6 : Float(16)
      %7 : Long()) {
  %8 : Float(1, 16, 30, 30) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[0, 0, 0, 0], strides=[1, 1]](%0, %1, %2), scope: TestConv2d/Conv2d[conv2d]
  %9 : Float(1, 16, 30, 30) = onnx::BatchNormalization[epsilon=1e-05, is_test=1, momentum=1](%8, %3, %4, %5, %6), scope: TestConv2d/BatchNorm2d[bn_1]
  return (%9);
}

Graph inputs: ['0', '1', '2', '3', '4', '5', '6', '7']
Graph outputs: ['9']
State dict: ['conv2d.weight', 'conv2d.bias', 'bn_1.weight', 'bn_1.bias', 'bn_1.running_mean', 'bn_1.running_var', 'bn_1.num_batches_tracked']
 ____ 
graph node: TestConv2d/Conv2d[conv2d]
node id: 8
type: onnx::Conv
inputs: ['0', '1', '2']
outputs: ['TestConv2d/Conv2d[conv2d]']
name in state_dict: conv2d
attrs: {'dilations': [1, 1], 'group': 1, 'kernel_shape': [3, 3], 'pads': [0, 0, 0, 0], 'strides': [1, 1]}
is_terminal: False
Converting convolution ...
 ____ 
graph node: TestConv2d/BatchNorm2d[bn_1]
node id: 9
type: onnx::BatchNormalization
inputs: ['8', '3', '4', '5', '6']
outputs: ['TestConv2d/BatchNorm2d[bn_1]']
name in state_dict: bn_1
attrs: {'epsilon': 1e-05, 'is_test': 1, 'momentum': 1.0}
is_terminal: True
Converting batchnorm ...
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
~\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py in _create_c_op(graph, node_def, inputs, control_inputs)
   1575   try:
-> 1576     c_op = c_api.TF_FinishOperation(op_desc)
   1577   except errors.InvalidArgumentError as e:

InvalidArgumentError: Shape must be rank 1 but is rank 0 for 'bn_10.9351925092536912/cond/Reshape_4' (op: 'Reshape') with input shapes: [1,16,1,1], [].

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-1-7f985261f348> in <module>()
     24 
     25 # we should specify shape of the input tensor
---> 26 k_model = pytorch_to_keras(model, input_var, [(10, 32, 32,)], verbose=True)

~\Anaconda3\lib\site-packages\pytorch2keras\converter.py in pytorch_to_keras(model, args, input_shapes, change_ordering, training, verbose, names)
    313             node_input_names,
    314             layers, state_dict,
--> 315             names
    316         )
    317         if node_id in graph_outputs:

~\Anaconda3\lib\site-packages\pytorch2keras\normalization_layers.py in convert_batchnorm(params, w_name, scope_name, inputs, layers, weights, names)
     59             name=tf_name
     60         )
---> 61     layers[scope_name] = bn(layers[inputs[0]])
     62 
     63 

~\Anaconda3\lib\site-packages\keras\engine\base_layer.py in __call__(self, inputs, **kwargs)
    455             # Actually call the layer,
    456             # collecting output(s), mask(s), and shape(s).
--> 457             output = self.call(inputs, **kwargs)
    458             output_mask = self.compute_mask(inputs, previous_mask)
    459 

~\Anaconda3\lib\site-packages\keras\layers\normalization.py in call(self, inputs, training)
    204         return K.in_train_phase(normed_training,
    205                                 normalize_inference,
--> 206                                 training=training)
    207 
    208     def get_config(self):

~\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py in in_train_phase(x, alt, training)
   3121 
   3122     # else: assume learning phase is a placeholder tensor.
-> 3123     x = switch(training, x, alt)
   3124     if uses_learning_phase:
   3125         x._uses_learning_phase = True

~\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py in switch(condition, then_expression, else_expression)
   3056         x = tf.cond(condition,
   3057                     then_expression_fn,
-> 3058                     else_expression_fn)
   3059     else:
   3060         # tf.where needs its condition tensor

~\Anaconda3\lib\site-packages\tensorflow\python\util\deprecation.py in new_func(*args, **kwargs)
    452                 'in a future version' if date is None else ('after %s' % date),
    453                 instructions)
--> 454       return func(*args, **kwargs)
    455     return tf_decorator.make_decorator(func, new_func, 'deprecated',
    456                                        _add_deprecated_arg_notice_to_docstring(

~\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py in cond(pred, true_fn, false_fn, strict, name, fn1, fn2)
   2055     context_f = CondContext(pred, pivot_2, branch=0)
   2056     context_f.Enter()
-> 2057     orig_res_f, res_f = context_f.BuildCondBranch(false_fn)
   2058     if orig_res_f is None:
   2059       raise ValueError("false_fn must have a return value.")

~\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py in BuildCondBranch(self, fn)
   1893     """Add the subgraph defined by fn() to the graph."""
   1894     pre_summaries = ops.get_collection(ops.GraphKeys._SUMMARY_COLLECTION)  # pylint: disable=protected-access
-> 1895     original_result = fn()
   1896     post_summaries = ops.get_collection(ops.GraphKeys._SUMMARY_COLLECTION)  # pylint: disable=protected-access
   1897     if len(post_summaries) > len(pre_summaries):

~\Anaconda3\lib\site-packages\keras\layers\normalization.py in normalize_inference()
    165                     broadcast_gamma,
    166                     axis=self.axis,
--> 167                     epsilon=self.epsilon)
    168             else:
    169                 return K.batch_normalization(

~\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py in batch_normalization(x, mean, var, beta, gamma, axis, epsilon)
   1906             # so it may have extra axes with 1, it is not needed and should be removed
   1907             if ndim(mean) > 1:
-> 1908                 mean = tf.reshape(mean, (-1))
   1909             if ndim(var) > 1:
   1910                 var = tf.reshape(var, (-1))

~\Anaconda3\lib\site-packages\tensorflow\python\ops\gen_array_ops.py in reshape(tensor, shape, name)
   7432   if _ctx is None or not _ctx._eager_context.is_eager:
   7433     _, _, _op = _op_def_lib._apply_op_helper(
-> 7434         "Reshape", tensor=tensor, shape=shape, name=name)
   7435     _result = _op.outputs[:]
   7436     _inputs_flat = _op.inputs

~\Anaconda3\lib\site-packages\tensorflow\python\framework\op_def_library.py in _apply_op_helper(self, op_type_name, name, **keywords)
    785         op = g.create_op(op_type_name, inputs, output_types, name=scope,
    786                          input_types=input_types, attrs=attr_protos,
--> 787                          op_def=op_def)
    788       return output_structure, op_def.is_stateful, op
    789 

~\Anaconda3\lib\site-packages\tensorflow\python\util\deprecation.py in new_func(*args, **kwargs)
    452                 'in a future version' if date is None else ('after %s' % date),
    453                 instructions)
--> 454       return func(*args, **kwargs)
    455     return tf_decorator.make_decorator(func, new_func, 'deprecated',
    456                                        _add_deprecated_arg_notice_to_docstring(

~\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py in create_op(***failed resolving arguments***)
   3153           input_types=input_types,
   3154           original_op=self._default_original_op,
-> 3155           op_def=op_def)
   3156       self._create_op_helper(ret, compute_device=compute_device)
   3157     return ret

~\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py in __init__(self, node_def, g, inputs, output_types, control_inputs, input_types, original_op, op_def)
   1729           op_def, inputs, node_def.attr)
   1730       self._c_op = _create_c_op(self._graph, node_def, grouped_inputs,
-> 1731                                 control_input_ops)
   1732 
   1733     # Initialize self._outputs.

~\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py in _create_c_op(graph, node_def, inputs, control_inputs)
   1577   except errors.InvalidArgumentError as e:
   1578     # Convert to ValueError for backwards compatibility.
-> 1579     raise ValueError(str(e))
   1580 
   1581   return c_op

ValueError: Shape must be rank 1 but is rank 0 for 'bn_10.9351925092536912/cond/Reshape_4' (op: 'Reshape') with input shapes: [1,16,1,1], [].

Multiple Inputs

Since my forward function in PyTorch has three inputs, I was wondering if the pytorch2keras converter can handle multiple variables? Does the converter support tuple inputs?

For example:
x is a (48,15)
y is (48,)
z is a tuple of two of these --> (2, 48, 128)

from converter import pytorch_to_keras
---> we should specify shape of the input tensor
k_model = pytorch_to_keras(model, (x, y, z), (???), verbose=True)

Test MENet / Inception v3 with a new version of the converter.

Add new models to the test set.

TypeError: forward() missing 2 required positional arguments: 'y' and 'z'

When I try the tests/multiple_inputs.py, it reports the following errors (I use pytorch 0.4):

TypeError: forward() missing 2 required positional arguments: 'y' and 'z'

What could be the problem? Thanks.

couldn't install the tools

when I do pip install pytorch2keras in my virtual environment “conda，python3.6” I got an error:
`Collecting pytorch2keras
Downloading https://files.pythonhosted.org/packages/2f/61/9fd96f62814ddf7d0a3d60db85be6965451c7d6efd8d5c52b1877db68bd3/pytorch2keras-0.1.6.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "/home/yx.wang/anaconda3/envs/py3torch2keras/lib/python3.6/site-packages/pip/_internal/download.py", line 436, in get_file_content
with open(url, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'requirements.txt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-install-zk3qnkav/pytorch2keras/setup.py", line 16, in <module>
    reqs = [str(ir.req) for ir in install_reqs]
  File "/tmp/pip-install-zk3qnkav/pytorch2keras/setup.py", line 16, in <listcomp>
    reqs = [str(ir.req) for ir in install_reqs]
  File "/home/yx.wang/anaconda3/envs/py3torch2keras/lib/python3.6/site-packages/pip/_internal/req/req_file.py", line 82, in parse_requirements
    filename, comes_from=comes_from, session=session
  File "/home/yx.wang/anaconda3/envs/py3torch2keras/lib/python3.6/site-packages/pip/_internal/download.py", line 440, in get_file_content
    'Could not open requirements file: %s' % str(exc)
pip._internal.exceptions.InstallationError: Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-zk3qnkav/pytorch2keras/
`

High max error

Ran /tests/resnet50.py and /tests/alexnet.py

For resnet50 I got this error

1.88754e+08
Max error: 1079778560.0

For alexnet I got this error

0.062452
Max error: 0.11586721986532211

Is this expected behavior, or is there anything wrong with my versions / set-up?

Implement recurrent layers

Add support of recurrent layers to the converter:

GRU
LSTM

Error in AlexNet test

Describe the bug
When running the provided test for AlexNet, encountered KeyError: 'classifier.0.weight'

I checked what keys does weights dict have:

odict_keys(['features.0.weight', 'features.0.bias', 'features.3.weight', 'features.3.bias', 'features.6.weight', 'features.6.bias', 'features.8.weight', 'features.8.bias', 'features.10.weight', 'features.10.bias', 'classifier.1.weight', 'classifier.1.bias', 'classifier.4.weight', 'classifier.4.bias', 'classifier.6.weight', 'classifier.6.bias'])

To Reproduce
Simply run pytorch2keras/tests/models/alexnet.py

Logs
....
....
....
Converting flatten ...

graph node: AlexNet/Sequential[classifier]/Dropout[0]
node id: 104
type: onnx::Dropout
inputs: ['101']
outputs: ['AlexNet/Sequential[classifier]/Dropout[0]', 'AlexNet/Sequential[classifier]/Dropout[0]']
name in state_dict: classifier.0
attrs: {'ratio': 0.5}
is_terminal: False
Converting dropout ...

graph node: AlexNet/Sequential[classifier]/Dropout[0]
node id: 109
type: onnx::Gemm
inputs: ['104', 'input11', 'input12']
outputs: ['AlexNet/Sequential[classifier]/Dropout[0]']
name in state_dict: classifier.0
attrs: {'alpha': 1.0, 'beta': 1.0, 'transB': 1}
is_terminal: False
Converting Linear ...
Traceback (most recent call last):
File "validate_cnn.py", line 39, in
k_model = pytorch_to_keras(model, input_var, (3, 224, 224,), verbose=True)
File "C:\Users\xxxxxx\work\WPy-3670\python-3.6.7.amd64\lib\site-packages\pytorch2keras\converter.py", line 325, in pytorch_to_keras
names
File "C:\Users\xxxxxx\work\WPy-3670\python-3.6.7.amd64\lib\site-packages\pytorch2keras\linear_layers.py", line 34, in convert_gemm
W = weights[weights_name].numpy().transpose()
KeyError: 'classifier.0.weight'

Environment (please complete the following information):

OS: Windows 7
Python 3.6, PyTorch 1.0
Latest pytorch2keras master

Possible feature expansion for concat layer

Today I was struggling with converting my inception-based model.
Apart from the padding hack, I noticed a potential feature for your library.

You assume that only 2 inputs are concatenated into 1, but it case of my model I have 2 and 4. I guess in a more general case you may consider including a proper variation of the below modification into future versions:

def convert_concat(node, node_name, input_names, output_name, layers):
    """
    Convert concatenation.

    Args:
        node: pytorch node element.
        node_name: pytorch node name
        input_name: pytorch input node name
        output_name: pytorch output node name
        layers: dictionary with keras tensors
    """
    print('Conerting concat ...')
    
    # basically a very shitty hack because my model contains only 2 or 4 concats
    if len(input_names)==2:
        model0 = layers[input_names[0]]
        model1 = layers[input_names[1]]
        cat = keras.layers.Concatenate(name=output_name, axis=node.dim)
        layers[output_name] = cat([model0, model1])    
    elif len(input_names)==4:
        model0 = layers[input_names[0]]
        model1 = layers[input_names[1]]
        model2 = layers[input_names[2]]
        model3 = layers[input_names[3]]   
        cat = keras.layers.Concatenate(name=output_name, axis=node.dim)
        layers[output_name] = cat([model0, model1, model2, model3])

Problem with Conv2d 1 channel -> several channels

Hi, thanks for great job.

I found particular problem:
when converting module of a type nn.Conv2d(1,output_channel,3,stride=(1,1),bias=True),
where output_channel >1 and groups =1,
in layers.py it is assumed that we deal with depthwise separable convolution (line 100),
and I got the following error (I changed input image from 3x320x320 to 1x320x320 in mobilinet.py):

File "mobilinet.py", line 136, in
k_model = pytorch_to_keras(model, input_var, (1, 224, 224,), verbose=True)
File "C:\ProgramData\Miniconda3\lib\site-packages\pytorch2keras-0.1.11-py3.5.egg\pytorch2keras\converter.py", line 314, in pytorch_to_keras
names
File "C:\ProgramData\Miniconda3\lib\site-packages\pytorch2keras-0.1.11-py3.5.egg\pytorch2keras\layers.py", line 147, in convert_conv
layers[scope_name] = conv(layers[input_name])
File "C:\ProgramData\Miniconda3\lib\site-packages\keras\engine\base_layer.py", line 436, in call
self.set_weights(self._initial_weights)
File "C:\ProgramData\Miniconda3\lib\site-packages\keras\engine\base_layer.py", line 1058, in set_weights
'provided weight shape ' + str(w.shape))
ValueError: Layer weight shape (3, 3, 1, 1) not compatible with provided weight shape (3, 3, 320, 1)

I temporally fixed this by changing line 100 to:
if n_groups == in_channels and n_groups>1:

KeyError: 'dropout.weight'

I have encounterd a new problem when I tried to run the code as follows:

import torch.nn as nn
from pytorch2keras.converter import pytorch_to_keras
import numpy as np
import torch
from torch.autograd import Variable

class CifarNet(nn.Module):
    def __init__(self):
        super(CifarNet, self).__init__()
        self.conv1 = nn.Conv2d(3, 64, kernel_size=3)
        self.conv2 = nn.Conv2d(64, 64, kernel_size=3)
        self.conv3 = nn.Conv2d(64, 128, kernel_size=3)
        self.conv4 = nn.Conv2d(128, 128, kernel_size=3)

        self.pool = nn.MaxPool2d(2, 2)

        self.relu = nn.ReLU(inplace=True)
        self.fc1 = nn.Linear(3200, 256)
        
        self.dropout = nn.Dropout(0.5)
        
        self.fc2 = nn.Linear(256, 256)
        self.fc3 = nn.Linear(256, 10)

    def forward(self, x):
        x = self.relu(self.conv1(x))
        x = self.relu(self.conv2(x))
        x = self.pool(x)

        x = self.relu(self.conv3(x))
        x = self.relu(self.conv4(x))
        x = self.pool(x)

        x = x.view(-1, 3200)
        x = self.relu(self.fc1(x))
        x = self.dropout(x)
        
        x = self.relu(self.fc2(x))
        x = self.fc3(x)

        return x

# Load the trained model from file
trained_model = CifarNet()
trained_model.load_state_dict(torch.load('results/cifar_best.pth'))

    
input_np = np.random.uniform(0, 1, (1, 3, 32, 32))
input_var = Variable(torch.FloatTensor(input_np))
k_model = pytorch_to_keras(trained_model, input_var, [(3, 32, 32,)], verbose=True)

which reported as follows:

Traceback (most recent call last):
  File "pytorch4keras.py", line 40, in <module>
    k_model = pytorch_to_keras(trained_model, input_var, [(3, 32, 32,)], verbose=True)
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/site-packages/pytorch2keras/converter.py", line 332, in pytorch_to_keras
    names
  File "/root/anaconda3/envs/python3_lth2/lib/python3.6/site-packages/pytorch2keras/linear_layers.py", line 34, in convert_gemm
    W = weights[weights_name].numpy().transpose()
KeyError: 'dropout.weight'

Is the dropout layer not include in the converter framework ?

Lambda layers in the keras model

Continued test from issue #41, it seems like the exported keras model has lambda op. Is it correct?

output of print (k_model.summary()) in #41

Layer (type)                 Output Shape              Param #   
=================================================================
input0 (InputLayer)          (None, 3, 224, 224)       0         
_________________________________________________________________
lambda_1 (Lambda)            (None, 3, 232, 232)       0         <-- lambda layer
_________________________________________________________________
conv1.conv2d0.19608361958321 (None, 32, 224, 224)      7808      
_________________________________________________________________
in10.5890283736035935 (Lambd (None, 32, 224, 224)      0         <-- lambda layer
_________________________________________________________________
relu0.8399978862622054 (Acti (None, 32, 224, 224)      0         
=================================================================
Total params: 7,808
Trainable params: 7,808
Non-trainable params: 0

Trying to make the generated keras model file to tfjs (which does not support lambda layer). Knowing that lambda can be any python codes. This would make it not portable to non-python frameworks, such as c++ or js based ones.

Don't know if it's logical to ask, but is it possible to workaround this requirements for lambda layer?

Does batchnorm converting work properly?

First of all thanks for this amazing converter, great jog ;)
So I have latest versions of pytorch, keras and this converter. And it seems like batch normalization layer isn't converted properly. Do you know why? Do you have the same issue?)

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.