jakc4103 / dfq Goto Github PK

Thx for your great job ! but I have some questions about your code:

1. the fused model(mobilenetv2) loss 1% acc compare to the fp model . In the main_cls.py , you do switch_layers() before merge_batchnorm(), after the switch_layer() the QuantNconv2d() quantize the input to 8bits by default ? I'm confused about it
2. in the merge_batchnorm(), you save the torch.abs( bn gamma) in the 'fake weight' buffer, why not bn gamma ?

Hi,
I managed to work around it but here another error I have:

AttributeError: 'QuantNConv2d' object has no attribute 'quant'

Will glad to know how to solve it.

Hi, thank you for sharing your work.

I am trying to test this out on the ResNet model.
I am using the pre-trained resnet18 on torchvision. Without any options, the model will reach its reported accuracy as expected, but with the equalization and quantization, its accuracy drops to 0.1%.

Is this an expected behavior due to specific methods in the implementation? If so, would you kindly help me out in fixing this?

Thanks.

(env) chaahm@HCAPC3612:~/DFQ$ python main_ssd.py --quantize --relu --equalize
main_ssd.py:90: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
all_gt_boxes[class_index][image_id] = torch.tensor(all_gt_boxes[class_index][image_id])
register <class 'torch.nn.modules.conv.Conv2d'> <class 'utils.quantize.QuantNConv2d'>
register <class 'torch.nn.modules.linear.Linear'> <class 'utils.quantize.QuantNLinear'>
register <class 'torch.nn.modules.activation.ReLU6'> <class 'torch.nn.modules.activation.ReLU'>
Traceback (most recent call last):
File "main_ssd.py", line 214, in
net, transformer = switch_layers(net, transformer, data, module_dict, ignore_layer=[QuantMeasure], quant_op=args.quantize)
File "/home/chaahm/DFQ/utils/layer_transform.py", line 158, in switch_layers
transformer._build_graph(model, data) # construt graph after all state_dict loaded
File "/home/chaahm/DFQ/PyTransformer/transformers/torchTransformer.py", line 349, in _build_graph
self.log = tmp_model.forward(self.log)
File "/home/chaahm/DFQ/modeling/detection/ssd.py", line 58, in forward
x = layer(x)
File "/home/chaahm/env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/home/chaahm/env/lib/python3.7/site-packages/torch/nn/modules/container.py", line 100, in forward
input = module(input)
File "/home/chaahm/env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/home/chaahm/DFQ/PyTransformer/transformers/utils.py", line 330, in forward
out_tensor = self.origin_layer(log_tensor).clone()
File "/home/chaahm/env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/home/chaahm/DFQ/utils/quantize.py", line 246, in forward
input = self.quant(input)
File "/home/chaahm/env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 594, in getattr
type(self).name, name))
AttributeError: 'QuantNConv2d' object has no attribute 'quant'

Hi,
I have trace your code, but not find execution of quantize weight.

In main_cls.py, set option such --quantize, --equalize is enable with mobilenetv2. When program is done two function of cross_layer_equalization() and set_quant_minmax(), observe the forward of this function QuantNConv2d during inference, self.quant(input) is doing like fake-quantization, result is float, Then just run conv2d(), there is no quantization of weight in the process.

So my question is is there no quantization weight in this code?

quantize.py :

class QuantNConv2d(nn.Conv2d):
    """docstring for QuantNConv2d."""

    def __init__(self, in_channels, out_channels, kernel_size,
                 stride=1, padding=0, dilation=1, groups=1, bias=True, num_bits=8, momentum=0.1):
        super(QuantNConv2d, self).__init__(in_channels, out_channels, kernel_size,
                                      stride, padding, dilation, groups, bias)
        self.quant = QuantMeasure(num_bits=num_bits, momentum=momentum)


    def forward(self, input):
        input = self.quant(input)

        output = F.conv2d(input, self.weight, self.bias, self.stride,
                            self.padding, self.dilation, self.groups)

        return output

Hi,

Thanks for your fantastic codebase. I have a question regarding how layer pairs are formed in cross layer equalization. Let me take Layer1 -> Layer2 -> Layer3 -> Layer4 as an example. After reading the paper, I thought the layer pairs are (1,2), (2,3), (3,4). However, in your code, the cross-layer pairs are (1,2), (3,4). Do I misunderstand the paper or your code?

Thanks!

QuantNConv2d will be use if trainale if false.

But I don't see weigth quantization operator in the layer.
When trainable is false, only quantize input？

I have come across an unexpected problem when I run with "python3 main_cls.py --quantize --relu --equalize",

load weight: modeling/classification/mobilenetv2_1.0-f2a8633.pth.tar
register <class 'torch.nn.modules.activation.ReLU6'> <class 'torch.nn.modules.activation.ReLU'>
register <class 'torch.nn.modules.conv.Conv2d'> <class 'utils.quantize.QuantNConv2d'>
register <class 'torch.nn.modules.linear.Linear'> <class 'utils.quantize.QuantNLinear'>
Traceback (most recent call last):
File "main_cls.py", line 214, in
main()
File "main_cls.py", line 132, in main
model, transformer = switch_layers(model, transformer, data, module_dict, ignore_layer=[QuantMeasure], quant_op=args.quantize)
File "/home/notebook/code/personal/80296970/DFQ/utils/layer_transform.py", line 157, in switch_layers
model = transformer.trans_layers(model, update=True if key == 1 else False)
File "/home/notebook/code/personal/80296970/DFQ/PyTransformer/transformers/torchTransformer.py", line 84, in trans_layers
_layer_new = self._register_dicttype(getattr(model, module_name)) # only give positional args
File "/home/notebook/code/personal/80296970/DFQ/utils/quantize.py", line 347, in init
super(QuantNLinear, self).init(in_features, out_features, bias)
File "/home/notebook/code/personal/80296970/dnn-quant-ocs/env/lib/python3.5/site-packages/torch/nn/modules/linear.py", line 72, in init
self.weight = Parameter(torch.Tensor(out_features, in_features))
TypeError: new() received an invalid combination of arguments - got (NoneType, NoneType), but expected one of:

• (torch.device device)
• (torch.Storage storage)
• (Tensor other)
• (tuple of ints size, torch.device device)
  didn't match because some of the arguments have invalid types: (NoneType, NoneType)
• (object data, torch.device device)
  didn't match because some of the arguments have invalid types: (NoneType, NoneType)

Have you ever meet this problem and what is the solution? Thank you.

Hi Jakc,
Thank you for your implementation!
I'm trying to reimplement the data free quantization and I'm confused about bias absorbing.
In the implementation of bias absorption with batchnorm layer, I think bn1.bias -= c, and bn2.bias += conv2.weight*bn2.weight/sqrt(bn2.var)*c, However, in your code, I saw bn2.bias += conv2.weight*c.
I'm wondering why you did not add bn2.weight/sqrt(bn2.var). And here is my implementation:

def bias_absorption(W1, W2, B1, B2, V2, G1, G2, N=3): # W:weight, B:bias, V:variance, G:gamma
    c = B1-N*G1 # beta-3*gamma
    c = c.clamp(0)

    Wc = torch.zeros(W2.shape[0])
    if(W1.shape[0]==W2.shape[1]): # pixelwise convolution
        for i in range(W2.shape[0]):
            Wc[i] = torch.sum(W2[i].squeeze()*c)*G2[i]/torch.sqrt(V2[i])
    else:                         # depthwise convolution
        for i in range(W2.shape[0]):
            Wc[i] = torch.sum(W2[i])*c[i]*G2[i]/torch.sqrt(V2[i])
    for i in range(len(c)):
        B1[i] = B1[i] - c[i]
    for i in range(len(Wc)):
        B2[i] = B2[i] + Wc[i]
    return c

In the paper, the author gives the derivation of c by the following equations,

and thus

However, the equation above neglect the batchnorm, if we take batchnorm into account, we will have:

where $\mathbf{W},\gamma, \beta, \mu, \operatorname{Var}$ is the origin paramter of convolution and batchnorm.

For ease of description, the equation above is reduced to

Thus, we have

Thank you very much!

Some Mobilenet v1 utilize ReLU6 as the activation function. If I replaced ReLU6 with ReLU, the accuracy drop is obvious even I didn't do any compensation and quantization. Do you have any comment on such situation?

I tried

 python3 main_cls.py --quantize --relu --equalize --correction

However, I got only 0.004945164506480558 accuracy
The log is given below:

jiangwx@SARS-AORUS:~/DFQ$ python3 main_cls.py --quantize --relu --equalize --correction
load weight: modeling/classification/mobilenetv2_1.0-f2a8633.pth.tar
register <class 'torch.nn.modules.conv.Conv2d'> <class 'utils.quantize.QuantNConv2d'>
register <class 'torch.nn.modules.linear.Linear'> <class 'utils.quantize.QuantNLinear'>
register <class 'torch.nn.modules.activation.ReLU6'> <class 'torch.nn.modules.activation.ReLU'>
Setting num_bits for targ layers...
Start cross layer equalization
Start bias correction
Quantizing Layer parameters
SET QUANT MIN MAX
Start inference
Acc: 0.004945164506480558

I noticed that you adjusted weight and bias of batchnorm layer by multiplying them by s

in file main_cls.py

Hi,

In mobilenetv2_ssd, how do you cross conv branches layer equalization, e.g this figure

I noticed that cross layer equalization happens before weight quantization

Thanks for your work, and I notice DFQ will work well in detect task, but when i transfer DFQ to my facedetect task, the result will got nothing. So i want to know the migration of DFQ?
@jakc4103

Hi, It is kind of you to share your work.
Very appreciate it.

I have been trying to run this repo mainly layer equalization. However, I've encountered some troubles. The accuracy I keep getting is original model accuracy, as is graph doesn't reflect on the model.
In main_cls.py, I set options to --relu --equaliz --correction
Do you have any idea why I keep getting this behavior?

I cloned the repo and tried to run the demo but was having this error:

(env) chaahm@HCAPC3612:~/DFQ$ python main_cls.py --quantize --relu --equalize
load weight: modeling/classification/mobilenetv2_1.0-f2a8633.pth.tar
register <class 'torch.nn.modules.conv.Conv2d'> <class 'utils.quantize.QuantNConv2d'>
register <class 'torch.nn.modules.linear.Linear'> <class 'utils.quantize.QuantNLinear'>
Traceback (most recent call last):
File "main_cls.py", line 214, in
main()
File "main_cls.py", line 132, in main
model, transformer = switch_layers(model, transformer, data, module_dict, ignore_layer=[QuantMeasure], quant_op=args.quantize)
File "/home/chaahm/DFQ/utils/layer_transform.py", line 157, in switch_layers
model = transformer.trans_layers(model, update=True if key == 1 else False)
File "/home/chaahm/DFQ/PyTransformer/transformers/torchTransformer.py", line 84, in trans_layers
_layer_new = self._register_dicttype(getattr(model, module_name)) # only give positional args
File "/home/chaahm/DFQ/utils/quantize.py", line 347, in init
super(QuantNLinear, self).init(in_features, out_features, bias)
File "/home/chaahm/env/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 72, in init
self.weight = Parameter(torch.Tensor(out_features, in_features))
TypeError: new() received an invalid combination of arguments - got (NoneType, NoneType), but expected one of:

• (torch.device device)
• (torch.Storage storage)
• (Tensor other)
• (tuple of ints size, torch.device device)
  didn't match because some of the arguments have invalid types: (NoneType, NoneType)
• (object data, torch.device device)
  didn't match because some of the arguments have invalid types: (NoneType, NoneType)

Thank you for sharing the code!
Does the code match the PyTransformer main branch version?
It throws an error at runtime, saying that the number of arguments in the transformer's _build_graph function is incorrect.

Hi, I want to know how to handle this situation exactly, I have limited ability to read the code..
c. many to many. ex: BatchNorm1->ReLU->
------------------------------------------------------QuantAdd->
------------------------------ BatchNorm2->ReLU->-----;---------QuantAdd -> input
---------------------------------------------- BatchNorm3->ReLU->
Does it mean the range of input is [0,beta+n×gamma]?
And beta=(BatchNorm1.beta+BatchNorm2.beta+BatchNorm3.beta)/3
gamma=(BatchNorm1.gamma+BatchNorm2.gamma+BatchNorm3.gamma)/3

Hi, It is kind of you to share your work.
Very appreciate it.

I had used the model in the modeling folder. It works very well.
However, I've encountered some troubles when applying the method on Torchvision pre-trained MobilenetV2.

1. Both --absorption, and --correction,
   It seems that some layer in the graph don't have the attribute "bias".
   Actually, it got a "None".

2. I've skipped the problem above by if-else.
   Result of the model in the modeling folder remains the same.
   Back to Torchvision pre-trained MobilenetV2,
   it just like the closed issue #10
   when --equalize, the accuracy drop tremendously to 0.1%
   No matter with --absorption, --correction, --distill_range.

Do these outputs to be expected since specific methods in the implementation?
If so, would you like to helping me finding out the reason?

Thank you.

Hi jakc4103,
Thanks a lot for your excellent code! In previous issues, you mentioned the bias absorption part did not work. Did you mean the absorption part reduced accuracy or just made no difference? Does bias absorption part work now?

Hi ,
When I try to run the main.py , the error in the title occured.Did you change the file "PyTransformer.transformers.quantize" which it is not in the version "6d1900b"?

sorry ,I found it. >_<

Hi.

Thanks for the great work.

I tried running the main_cls.py with relu equilize and correction true for InceptionV3 (from PytorchCV model zoo), and got about 0.002 accuracy, whereas its working fine for resnet18, resnet50 etc.

Note: M using the input size 299 for inceptionv3.

Appreciate your comment.

Thanks !

I believe the code here Here should be repaced with graph[idx].quant = QuantMeasure(num_bits=bits_activation). As it is missing the proper argument name, it will always add a QuantMeasure module with 8-bit acts.

We found it very useful to cal min and max of input data by set_quant_minmax, instead of calibrating.
But the logic is not very clearly, can you explain it in detail?
Appreciate for your help~

Hi,
Thanks for sharing!
any idea what is the reason for the following error:
File "DFQ/PyTransformer/transformers/utils.py", line 177, in getattr
return object.getattribute(self, name)
AttributeError: 'Log' object has no attribute 'getRecordTensorOP'

Thanks!

Hi there, very impressive work!
While implementing your project, I have a little trouble compiling the inference_cls.cpp using the CMakeLists.txt.

Here is my CMakeLists.txt:

cmake_minimum_required(VERSION 2.8.12)
project(dfq)
include_directories(/Users/heyiqian/Desktop/EdgeAI/ncnn/src)
include_directories(/Users/heyiqian/Desktop/EdgeAI/ncnn/build/src)

#openmp
FIND_PACKAGE( OpenMP REQUIRED)
if(OPENMP_FOUND)
message("OPENMP FOUND")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${OpenMP_C_FLAGS}")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS}")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${OpenMP_EXE_LINKER_FLAGS}")
endif()

#ncnn
set(NCNN_LIBS /Users/heyiqian/Desktop/EdgeAI/ncnn/build/install/lib/libncnn.a)
set(NCNN_INCLUDE_DIRS /Users/heyiqian/Desktop/EdgeAI/ncnn/build/install/include)
include_directories(${NCNN_INCLUDE_DIRS})

#opencv
find_package( OpenCV REQUIRED )
include_directories( ${OpenCV_INCLUDE_DIRS} )

add_executable(inference_cls inference_cls.cpp)
target_link_libraries(inference_cls ${NCNN_LIBS})
target_link_libraries( inference_cls ${OpenCV_LIBS} )
target_link_libraries(${PROJECT_NAME} ${OpenMP_CXX_LIBRARIES})

Here is my cmake command:

cmake -DCMAKE_C_COMPILER="/usr/local/Cellar/llvm/11.1.0/bin/clang" ->
DCMAKE_CXX_COMPILER="/usr/local/Cellar/llvm/11.1.0/bin/clang++" ..

And this is what I get from the CMakeError.log(parts of it, others are similar to the following ones):

Detecting C OpenMP failed with the following output:
Change Dir: /Users/heyiqian/Desktop/EdgeAI/DFQ-master/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_367b0/fast && /Library/Developer/CommandLineTools/usr/bin/make -f CMakeFiles/cmTC_367b0.dir/build.make CMakeFiles/cmTC_367b0.dir/build
Building C object CMakeFiles/cmTC_367b0.dir/OpenMPTryFlag.c.o
/usr/local/Cellar/llvm/11.1.0/bin/clang -fopenmp=libomp -v -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -mmacosx-version-min=10.15 -o CMakeFiles/cmTC_367b0.dir/OpenMPTryFlag.c.o -c /Users/heyiqian/Desktop/EdgeAI/DFQ-master/build/CMakeFiles/FindOpenMP/OpenMPTryFlag.c
clang version 11.1.0
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /usr/local/Cellar/llvm/11.1.0/bin
(in-process)
"/usr/local/Cellar/llvm/11.1.0/bin/clang-11" -cc1 -triple x86_64-apple-macosx10.15.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name OpenMPTryFlag.c -mrelocation-model pic -pic-level 2 -mframe-pointer=all -fno-rounding-math -munwind-tables -target-sdk-version=11.0 -fcompatibility-qualified-id-block-type-checking -target-cpu penryn -debugger-tuning=lldb -target-linker-version 609.8 -v -resource-dir /usr/local/Cellar/llvm/11.1.0/lib/clang/11.1.0 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/local/include -internal-isystem /usr/local/Cellar/llvm/11.1.0/lib/clang/11.1.0/include -internal-externc-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/include -fdebug-compilation-dir /Users/heyiqian/Desktop/EdgeAI/DFQ-master/build/CMakeFiles/CMakeTmp -ferror-limit 19 -fopenmp -fopenmp-cuda-parallel-target-regions -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fmax-type-align=16 -o CMakeFiles/cmTC_367b0.dir/OpenMPTryFlag.c.o -x c /Users/heyiqian/Desktop/EdgeAI/DFQ-master/build/CMakeFiles/FindOpenMP/OpenMPTryFlag.c
clang -cc1 version 11.1.0 based upon LLVM 11.1.0 default target x86_64-apple-darwin19.6.0
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/local/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
/usr/local/Cellar/llvm/11.1.0/lib/clang/11.1.0/include
/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/include
/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks (framework directory)
End of search list.
Linking C executable cmTC_367b0
/usr/local/Cellar/cmake/3.19.6/bin/cmake -E cmake_link_script CMakeFiles/cmTC_367b0.dir/link.txt --verbose=1
/usr/local/Cellar/llvm/11.1.0/bin/clang -fopenmp=libomp -v -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -mmacosx-version-min=10.15 -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/cmTC_367b0.dir/OpenMPTryFlag.c.o -o cmTC_367b0 -v -Wl,-v
clang version 11.1.0
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /usr/local/Cellar/llvm/11.1.0/bin
"/usr/bin/ld" -demangle -lto_library /usr/local/Cellar/llvm/11.1.0/lib/libLTO.dylib -dynamic -arch x86_64 -platform_version macos 10.15.0 11.0 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -o cmTC_367b0 -search_paths_first -headerpad_max_install_names CMakeFiles/cmTC_367b0.dir/OpenMPTryFlag.c.o -v -lomp -lSystem /usr/local/Cellar/llvm/11.1.0/lib/clang/11.1.0/lib/darwin/libclang_rt.osx.a
@(#)PROGRAM:ld PROJECT:ld64-607.2
BUILD 04:30:50 Jul 14 2020
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
Library search paths:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/lib
Framework search paths:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/
ld: library not found for -lomp
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [cmTC_367b0] Error 1
make: *** [cmTC_367b0/fast] Error 2

Detecting CXX OpenMP failed with the following output:
Change Dir: /Users/heyiqian/Desktop/EdgeAI/DFQ-master/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_2a53f/fast && /Library/Developer/CommandLineTools/usr/bin/make -f CMakeFiles/cmTC_2a53f.dir/build.make CMakeFiles/cmTC_2a53f.dir/build
Building CXX object CMakeFiles/cmTC_2a53f.dir/OpenMPTryFlag.cpp.o
/usr/local/Cellar/llvm/11.1.0/bin/clang++ -Xclang -fopenmp -v -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -mmacosx-version-min=10.15 -o CMakeFiles/cmTC_2a53f.dir/OpenMPTryFlag.cpp.o -c /Users/heyiqian/Desktop/EdgeAI/DFQ-master/build/CMakeFiles/FindOpenMP/OpenMPTryFlag.cpp
clang version 11.1.0
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /usr/local/Cellar/llvm/11.1.0/bin
(in-process)
"/usr/local/Cellar/llvm/11.1.0/bin/clang-11" -cc1 -triple x86_64-apple-macosx10.15.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name OpenMPTryFlag.cpp -mrelocation-model pic -pic-level 2 -mframe-pointer=all -fno-rounding-math -munwind-tables -target-sdk-version=11.0 -fcompatibility-qualified-id-block-type-checking -target-cpu penryn -debugger-tuning=lldb -target-linker-version 609.8 -v -resource-dir /usr/local/Cellar/llvm/11.1.0/lib/clang/11.1.0 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -stdlib=libc++ -internal-isystem /usr/local/Cellar/llvm/11.1.0/bin/../include/c++/v1 -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/include/c++/v1 -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/local/include -internal-isystem /usr/local/Cellar/llvm/11.1.0/lib/clang/11.1.0/include -internal-externc-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/include -fdeprecated-macro -fdebug-compilation-dir /Users/heyiqian/Desktop/EdgeAI/DFQ-master/build/CMakeFiles/CMakeTmp -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fmax-type-align=16 -fopenmp -o CMakeFiles/cmTC_2a53f.dir/OpenMPTryFlag.cpp.o -x c++ /Users/heyiqian/Desktop/EdgeAI/DFQ-master/build/CMakeFiles/FindOpenMP/OpenMPTryFlag.cpp
clang -cc1 version 11.1.0 based upon LLVM 11.1.0 default target x86_64-apple-darwin19.6.0
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/include/c++/v1"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/local/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
/usr/local/Cellar/llvm/11.1.0/bin/../include/c++/v1
/usr/local/Cellar/llvm/11.1.0/lib/clang/11.1.0/include
/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/include
/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks (framework directory)
End of search list.
Linking CXX executable cmTC_2a53f
/usr/local/Cellar/cmake/3.19.6/bin/cmake -E cmake_link_script CMakeFiles/cmTC_2a53f.dir/link.txt --verbose=1
/usr/local/Cellar/llvm/11.1.0/bin/clang++ -Xclang -fopenmp -v -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -mmacosx-version-min=10.15 -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/cmTC_2a53f.dir/OpenMPTryFlag.cpp.o -o cmTC_2a53f -v -Wl,-v
clang version 11.1.0
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /usr/local/Cellar/llvm/11.1.0/bin
"/usr/bin/ld" -demangle -lto_library /usr/local/Cellar/llvm/11.1.0/lib/libLTO.dylib -dynamic -arch x86_64 -platform_version macos 10.15.0 11.0 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -o cmTC_2a53f -search_paths_first -headerpad_max_install_names CMakeFiles/cmTC_2a53f.dir/OpenMPTryFlag.cpp.o -v -lc++ -lSystem /usr/local/Cellar/llvm/11.1.0/lib/clang/11.1.0/lib/darwin/libclang_rt.osx.a
@(#)PROGRAM:ld PROJECT:ld64-607.2
BUILD 04:30:50 Jul 14 2020
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
Library search paths:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/lib
Framework search paths:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/
Undefined symbols for architecture x86_64:
"_omp_get_max_threads", referenced from:
_main in OpenMPTryFlag.cpp.o
ld: symbol(s) not found for architecture x86_64
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [cmTC_2a53f] Error 1
make: *** [cmTC_2a53f/fast] Error 2

I've tried lots of methods from google to fix this, but none of them work.
Any suggestions may be helpful, looking forward for your reply!

Dear author:
Thanks for sharing this great work!
is this quantization technology able to quantize any pretrained cnn network? I tried your code and able to export to ncnn model. But I also intereseted in export it for coreml/MNN model. Could you give some tips?

Dear jakc,

Thank you for sharing your work of DFQ.
Very appreciate it!!

However, I have found that
the mAP of mobilnetv2 SSD_lite that https://github.com/qfgaohao/pytorch-ssd had claimed is 68.607.
And your result of original FP32 is 60.69.

Is there any reason that causes the phenomenon?
The fusion of batch-normal or something else?

Thank you.

Hi,
I am trying to reproduce the same results but in Keras, I have a couple of questions regarding the implementation

1. I was wondering in _layer_equalization function, shouldn't equation be 1 / s = (1 / r2) * sqrt(r1 * r2) instead of 1 / s = (1 / r1) * sqrt(r1 * r2) and range_1 = 2* torch.max(torch.abs(weight_first_group[ii])) instead of range_1 = torch.max(torch.abs(weight_first_group[ii]))
2. another thing when you create a relation graph, is it for every 2 layers which are connected, even if one nodes have 2 incoming connection, and of types convolution and depthwise convolution only?
3. Do the scales sometimes reach nan and diff infinity?

Thank you in advance

I have tested the base model of MobilnetV2, but the num_correct is always zero every batchs. The val dataset I use is ILSVRC2017_CLS-LOC, and the struction of val folder is
n02027492 n02123394 n02794156 n03417042 n03920288 n04423845 n13133613
n02028035 n02123597 n02795169 n03424325 n03924679 n04428191 n15075141
n02033041 n02124075 n02797295 n03425413 n03929660 n04429376 .....
Would you have any idea?

Hi. I want to get the result of weight symmetric quantization. So I set the symmetric=True in the function forward of class QuantConv2d, but it cause error
:AttributeError: 'float' object has no attribute 'cur_tensor'
So I change the code max_value = torch.max(torch.abs(max_value), torch.abs(min_value))
to max_value = max(abs(max_value), abs(min_value)), it's worked, but the acc result is zero. I don't understand why, do you have any idea?
I would be grateful if you could solve my question

Hi,
thanks a lot for the great work you have done here.
For ResNet18 the author of the paper wrote that the accuracy is 66.3% while here we get 58%, do you have any idea what the reason is
Thanks a lot :)

I am wondering how I can get the symmetric quantization (int8') result. I tried to set symmetric=True in the forward function if UniformQuantize class, however I got a different number. In RELU+LE case, I got 66.89. Any help will be appreciated.

Hi,

Thanks for the code. But with nothing changed, the code cannot run w/ the complaint: AttributeError: 'Tensor' object has no attribute 'putLayer'

1. do u have a requirment.txt so that we can follow your env?
2. The PyTransformer seems to be updating. Can you provide a folder (instead of a link) that works?

thanks a lot,
jun

here're the error output FYI.
Traceback (most recent call last):
File "/newDisk/users/junjiang/project_from_server/DFQ/main.py", line 177, in
orig_main()
File "/newDisk/users/junjiang/project_from_server/DFQ/main.py", line 91, in orig_main
model = switch_layers(model, transformer, data)
File "/newDisk/users/junjiang/project_from_server/DFQ/utils/layer_transform.py", line 194, in switch_layers
transformer._build_graph(model, data) # construt graph after all state_dict loaded
File "/newDisk/users/junjiang/project_from_server/DFQ/PyTransformer/transformers/torchTransformer.py", line 349, in _build_graph
self.log = tmp_model.forward(self.log)
File "/newDisk/users/junjiang/project_from_server/DFQ/modeling/deeplab.py", line 29, in forward
x, low_level_feat = self.backbone(input)
File "/home/junjiang/anaconda3/envs/DFQ/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/newDisk/users/junjiang/project_from_server/DFQ/modeling/backbone/mobilenet.py", line 119, in forward
low_level_feat = self.low_level_features(x)
File "/home/junjiang/anaconda3/envs/DFQ/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/junjiang/anaconda3/envs/DFQ/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/home/junjiang/anaconda3/envs/DFQ/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/junjiang/anaconda3/envs/DFQ/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/home/junjiang/anaconda3/envs/DFQ/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/newDisk/users/junjiang/project_from_server/DFQ/PyTransformer/transformers/utils.py", line 354, in forward
out_tensor = self.origin_layer(log_tensor).clone()
File "/home/junjiang/anaconda3/envs/DFQ/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/newDisk/users/junjiang/project_from_server/DFQ/PyTransformer/transformers/utils.py", line 349, in forward
cur_log.putLayer(self.origin_layer)
AttributeError: 'Tensor' object has no attribute 'putLayer'

Hi author,

Here is the result when I run the code:

• python main_cls.py --quantize --relu --equalize: 0.6865
• python main_cls.py --quantize --relu --equalize --correction: 0.68072

With training

• python main_cls.py --quantize --relu --equalize --trainable: 0.6873
• python main_cls.py --quantize --relu --equalize --trainable --correction: 0.68186

The accuracy dropped after bias correction. I took a look at the code, where the num_bits_bias is by default 16, which should be "Int8**" in the Readme.

Any suggestion on how could I make the accuracy after bias correction better? Thank you for the great work!

While I have the following command:
python main_ssd.py --quantize --correction --log --bits_bias 16

It shows me the warning message:
Warning: <class 'utils.quantize.QuantNConv2d'> layer before first batch norm layer detected. The calculated value range might be off.
Warning: <class 'utils.quantize.QuantNConv2d'> layer before first batch norm layer detected. The calculated value range might be off.
...

and the mAP is always below than your result, i.e., 0.6729297

Does the warning message lead to accuracy drop?

In many previous works, they show that it's more difficult to do quantization while the batch normalization is fused with convolution due to wider variance of weight range.
So, do you have any comment if I want to quantize the model with fused batch normalization and convolution?