Comments (9)
In the documentation we give some guidelines for debugging the performance of a model. I suggest you look at the MLIR representation of your network. I also strongly advise you use "rounding" when compiling.
from concrete-ml.
Hello,
Would it be possible to share your code (or at least a similar code that produces this exact error) ? Without more information on what you are trying to do, it is hard to understand the source of your issue 😄
Small tip: it is usually best to share code or errors using GitHub's code formatting section (either using `` or through the "Code" button available in the text edit toolbar right above) 😉 For example :
The following tensors were expected to be quantized, but the values found during calibration do not appear to be quantized.
from concrete-ml.
result.add_module('conv',qnn.QuantConv2d(in_channels=in_channels,out_channels=out_channels, kernel_size=kernel_size, stride=stride,padding=padding, weight_bit_width=bit, weight_quant=weight_quant, return_quant_tensor=True, bias=False))
result.add_module('bn', nn.BatchNorm2d(num_features=out_channels, eps=1e-4))
This is part of my model,I directly use BatchNorm2d layer without any other operations.What actions should I add to make it compile successfully?
from concrete-ml.
Hello again,
Would it be possible to give more code so that we can reproduce it by just copy-pasting it ? If could just be with fake data (but similar dtypes) for example.
For now it's still unclear why such issue would happen here, but you may try to set return_quant_tensor
to False in your QuantConv2d
! Not sure it'll fix it, but I'm interested in what this could do.
Also following my last small tip, you can display complete blocks of code framed by ``` ! For example :
result.add_module('conv',qnn.QuantConv2d(in_channels=in_channels,out_channels=out_channels, kernel_size=kernel_size, stride=stride,padding=padding, weight_bit_width=bit, weight_quant=weight_quant, return_quant_tensor=True, bias=False))
result.add_module('bn', nn.BatchNorm2d(num_features=out_channels, eps=1e-4))
from concrete-ml.
import brevitas
import brevitas.nn as qnn
import torch
import torch.nn as nn
from brevitas.quant import Int8ActPerTensorFloat, Int8WeightPerTensorFloat
def conv_bn(in_channels, out_channels, kernel_size, stride, padding,bit,
act_quant: brevitas.quant = Int8ActPerTensorFloat,
weight_quant: brevitas.quant = Int8WeightPerTensorFloat,):
result = nn.Sequential()
result.add_module('conv', qnn.QuantConv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=kernel_size, stride=stride,
padding=padding, weight_bit_width=bit, weight_quant=weight_quant, return_quant_tensor=True, bias=False))
result.add_module('bn', nn.BatchNorm2d(num_features=out_channels, eps=1e-4))
return result
class QuantmodelBlock(nn.Module):
def __init__(self, in_channels, out_channels ,stride: int,kernel_size: int, padding: int, bit: int,
act_quant: brevitas.quant = Int8ActPerTensorFloat,
weight_quant: brevitas.quant = Int8WeightPerTensorFloat,):
super(QuantmodelBlock, self).__init__()
self.in_channels = in_channels
self.bit = bit
padding_11 = padding - kernel_size // 2
self.nonlinearity = qnn.QuantReLU(return_quant_tensor=True, bit_width=bit, act_quant=act_quant)
self.rbr_identity = nn.BatchNorm2d(num_features=in_channels,eps=1e-4) if out_channels == in_channels and stride == 1 else None
self.rbr_dense = conv_bn(in_channels=in_channels, out_channels=out_channels, kernel_size=kernel_size, stride=stride,padding=padding, bit=bit)
print(in_channels, out_channels, kernel_size, stride, padding)
self.rbr_1x1 = conv_bn(in_channels=in_channels, out_channels=out_channels, kernel_size=1, stride=stride,padding=padding_11, bit=bit)
def forward(self, inputs):
if self.rbr_identity is None:
id_out = 0
else:
id_out = self.rbr_identity(inputs)
return self.nonlinearity(self.rbr_dense(inputs) + self.rbr_1x1(inputs) + id_out).value
class Quantmodel(nn.Module):
def __init__(self, num_blocks, width_multiplier,
bit: int,
output_size: int,
act_quant: brevitas.quant = Int8ActPerTensorFloat,
weight_quant: brevitas.quant = Int8WeightPerTensorFloat,):
super(Quantmodel, self).__init__()
assert len(width_multiplier) == 4
self.in_planes = min(16, int(16 * width_multiplier[0]))
self.stage0 = QuantmodelBlock(in_channels=3, out_channels=self.in_planes, kernel_size=3, padding=1, bit=bit, stride=1)
self.stage1 = self._make_stage(int(16 * width_multiplier[0]), num_blocks[0], stride=1, bit = bit)
self.stage2 = self._make_stage(int(32 * width_multiplier[1]), num_blocks[1], stride=1 ,bit = bit)
self.stage3 = self._make_stage(int(64 * width_multiplier[2]), num_blocks[2], stride=2, bit = bit)
self.stage4 = self._make_stage(int(64 * width_multiplier[3]), num_blocks[3], stride=2, bit = bit)
self.linear1 = qnn.QuantLinear(
in_features=int(64 * width_multiplier[3]),
out_features=output_size,
weight_quant=weight_quant,
weight_bit_width=bit,
bias=True,
return_quant_tensor=True,
)
self.identity = qnn.QuantIdentity(
bit_width=bit, act_quant=act_quant, return_quant_tensor=True
)
self.identity2 = qnn.QuantIdentity(
bit_width=bit, act_quant=act_quant, return_quant_tensor=True
)
self.identity3 = qnn.QuantIdentity(
bit_width=bit, act_quant=act_quant, return_quant_tensor=True
)
self.gap = nn.AvgPool2d(kernel_size=8, stride=1, padding=0, ceil_mode=False)
def _make_stage(self, planes, num_blocks, stride,bit):
strides = [stride] + [1]*(num_blocks-1)
blocks = []
for stride in strides:
blocks.append(QuantmodelBlock(in_channels=self.in_planes, out_channels=planes, stride=stride,bit=bit,kernel_size=3,padding=1))
self.in_planes = planes
return nn.ModuleList(blocks)
def forward(self, x):
out = self.identity(x)
out = self.stage0(out)
for stage in (self.stage1, self.stage2, self.stage3, self.stage4):
for block in stage:
out = block(out)
out = self.gap(out)
out = self.identity2(out)
out = torch.flatten(out, 1)
out = self.identity3(out)
out = self.linear1(out)
return out.value
def create_Quantmodel(bit: int, output_size: int):
return Quantmodel(num_blocks=[4, 4, 4, 1],width_multiplier=[1, 1, 1, 1], bit=bit, output_size=output_size)
Thanks,This is my complete model, and I hope to classify it on CIFAR. After completing the training, I compile it, but there was an error reported above.
from concrete-ml.
Great thanks a lot ! We'll take a look at it and get back to you once we know more 😉
from concrete-ml.
Quantization when doing tensor addition is a bit more tricky, my guess is that the error comes from
return self.nonlinearity(self.rbr_dense(inputs) + self.rbr_1x1(inputs) + id_out).value
Here you are adding tensors that must be quantized, yet you do not add QAT quantization for them. You need to do something like:
def __init__():
....
self.out_quant = qnn.QuantIdentity()
def forward(..):
....
return self.nonlinearity(self.out_quant(self.rbr_dense(inputs)) + self.out_quant(self.rbr_1x1(inputs)) + self.out_quant(id_out)).value
Alternatively you could have 3 different QuantiIdentity
for each of rbr_dense
, rbr_1x1
and id_out
parts. I'm not sure it will help, and having 3 of them that will slow down the execution.
from concrete-ml.
Thank you, I have another question. Where can I obtain some information about model encryption, such as the time or complexity required for encryption inference at each layer? Where can I see the information of the compiled graph? Are there any functions that I can use to get it?
from concrete-ml.
Thank you
from concrete-ml.
Related Issues (20)
- is independent of the actual bit precision limitation? HOT 4
- What is the difference between a graph and a circuit HOT 4
- Model dumping not working for Fully Connected Neural Network on MNIST Example HOT 2
- Docker image is not working on Apple M1/M2 chips HOT 11
- Can FHE inference run on cuda? HOT 1
- Accuracy mismatch between clear and FHE predictions in CIFAR-10 example HOT 4
- Can't emit artifacts HOT 3
- Does this library support running on GPU? HOT 7
- Certain ONNX Operators HOT 2
- Feature Request: support of torch.nn.Unfold function HOT 2
- Unknown attributes: count_include_pad when compiling with compile_torch_model HOT 2
- May I ask how to visualize the graph? HOT 1
- Quantized Unfold HOT 1
- How much RAM do I need? HOT 2
- When using the cifar-10 data set, the program is killed when it runs to the fhe_compatibility function HOT 6
- What is the difference between two bit? HOT 2
- WARNING: high error rate, more details with --display-optimizer-choice? HOT 16
- ImportError: cannot import name 'enc_split' from 'utils' in LLM case. HOT 2
- predict() got an unexpected keyword argument 'fhe' HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from concrete-ml.