Comments (7)
Hi, thanks again for your quick response!
In the end I decided to use QAT as you recommended and leave the other approach aside. I found the step-by-step guide very useful!!.
I therefore proceed to close the issue.
Thank you very much!!
from concrete-ml.
Hello again,
Thanks a lot for the new infos ! Just to make sure, did you apply any transformations on your inputs since you have trained your model ? For example, I can see that your inputs are flattened, but was that already the case when your model was being trained ? Be aware that inputs' shape should remain the same between fitting, compilation and prediction steps !
Besides, what error did you get when compiling with 6 bits ? If it is something like Cannot find crypto parameters
, this is most likely expected.
In fact, there's a few things to tell here. First, Concrete ML 0.6 does indeed handle 16 bits of precision, but setting n_bits=6
will most likely make your model exceed this constraint, especially with complex structures such as fully-connected neural networks. Such an overflow will make your model not being able to compile. If you want to know more about it, you can check our documentation about quantization, Neural Networks and our use of pruning !
Then, you might also want to convert your model to a Quantize Aware Training (QAT) one if you haven't done it yet ! The main reason for this is that neural networks perform much better under quantization constraints with such a technique. Besides, this conversion can easily be done using the Brevitas library ! You'll find more information in our step_by_step guide and how to import QAT models.
Finally, I invite you to check some of our examples if you haven't done it yet ! Among them, here's how we tackled the MNIST classification problem with our own built-in FC neural networks (which use QAT), reaching 97% accuracy with a few bits of quantization. We also give an example of how to solve this task with a custom model in one of our use case examples. If you are not able to train your model again using QAT, here's a more advanced example on how we succeeded to convert a pre-trained model to a QAT one and use it in Concrete ML for tackling the CIFAR datasets.
Let me know if you have more questions !
from concrete-ml.
Hello !
Great to see you experimenting with our Client-Server interface ! Based the error you get, I would say you probably have an issue with your inputs' shape. More precisely, I think your encrypted_input
's shape might not match the one used for your torch_inputset
's samples that you gave to compile_torch_model
.
However, it is a bit hard to know more. Would it be possible to give us access to your code or at least tell us more about what you are trying to achieve so that we can better understand the issue ?
Thanks !
from concrete-ml.
Hi, thank you very much for your quick response!!
I have been reviewing the dimensions, and it seems this is not the problem (even though the error raises that exception)
Trying some parameter combinations, I have compiled the torch model again but with 2 bits (I had n_bit=6
) (see the snippet below).
With this modification, I'm able to reproduce the original client-server
tutorial, but the accuracy is very low.
In the last release I though the number of bits was increased to 16 but I don't know why 6 bits did not work but everything seems ok with 2.
Code snippet
from concrete.ml.torch.compile import compile_torch_model
quantized_numpy_module = compile_torch_model(
torch_model=torch_fc_model, # this is the well-known mnist model,
torch_inputset=torch_inputset, # this tensor has the appropiate dimensions
n_bits=2)
Moreover, in another attempt to try the client-server
tutorial, I've realised that the compile_torch_model
function raises an error, depending on the combination of the torch_inputset
and the number of bits. I've tried setting as the torch_inputset
an array of the MNIST dataset ( torch_inputset = mnist_test_data[:-10]
), as well as a random array , both with the same vector dimensions:
input_output_feature = 784
input_shape = (input_output_feature,)
n_examples = 5000
# Create random input
torch_inputset = numpy.random.uniform(-100, 100, size=(n_examples, *input_shape))
Does the configuration of the number of bits depends on the size of the input?
Why does the whole tutorial work with 2 bits, but not with more than two?
Thanks a lot!!
from concrete-ml.
Hello again, thank you very much for your response!
I've been revising my code and I confirmed that my inputs are flattened in training, compilation and prediction, so same shapes are always used in each step.
When I try to use the function compile_torch_model
with 6 bits, the error I'm getting is:
RuntimeError
...
in compile_torch_model(torch_model, torch_inputset, import_qat, configuration, compilation_artifacts, show_mlir, n_bits, use_virtual_lib, p_error, global_p_error, verbose_compilation)
%21 = 0 # ClearScalar<uint1>
%22 = 63 # ClearScalar<uint6>
%23 = clip(%20, %21, %22) # EncryptedTensor<float64, shape=(1, 64)>
%24 = astype(%23, dtype=int_) # EncryptedTensor<uint1, shape=(1, 64)>
return %24
About converting the model to a QAT one, I thought this process was done in the function compile_torch_model since the output is a QuantizedModule. Am I forgetting to do an additional step? Is it preferable to use Brevitas?
Finally, I've tried the MNIST and CIFAR examples and they really get high accuracy so I will rely my next steps on these examples.
Thank you very much!
from concrete-ml.
Hello,
Thanks again for your feedback !
The error you give is not enough to determine the exact issue you're facing. Could you maybe share the full traceback ?
Besides, we only support models that are either already quantized using QAT or "float models" on which we then perform Post-Training Quantization (PTQ). If you call compile_torch_model
on your model without setting import_qat
to True, the function considers that your model falls into this second category and I believe this is your case. However, PTQ does not seem to work very well with Neural Networks, especially with small numbers of bits. Our built-in Neural Network models use QAT by default for this particular reason. If you want to use use QAT, I therefore invite you to follow our step-by-step guide on how you can indeed use Brevitas to do so !
It is great to know that you have been able to get good results with our examples. Let me know how it goes !
from concrete-ml.
Awesome 😄
If you ever need to ask some further questions or encounter other issues, we are also very active on FHE.org's discord server ("Concrete" channel). Similarly, you can reach out to us on our community forum, where many recurrent questions were already addressed. Don't forget to also star our repository if you find Concrete-ML interesting !
Have a nice day!
from concrete-ml.
Related Issues (20)
- [Question] Discord link in explanation HOT 2
- High accuracy variance during the training with SGDClassifier HOT 1
- Feature Request : Implement LogSoftmax, Softmax, ReduceMax HOT 3
- Performance Issues HOT 1
- Two consecutive Unsqueeze operations in QAT model throws error at compilation time HOT 2
- LLVM symbolizer error with LogisticRegression example HOT 15
- [Question] What HE algorithm is used? HOT 6
- [Question] AssertionError: Values must be float if value_is_float is set to True, got int64: [1] HOT 3
- AssertionError: Values must be float if value_is_float is set to True, got int64: [[[[102 14 188 ... 85 205 46] HOT 3
- RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! HOT 7
- Python 3.12 HOT 2
- Adding encrypted training for other ML models and DL models HOT 5
- quantized_module.forward() occured an error in "execute" mode HOT 8
- Feature request: Support Unfold torch operator HOT 10
- LLVM symbolizer error when running FHE in 'execute' mode HOT 7
- [Question] How does ReLU work in the new NN example HOT 6
- Feature request: support more padding method HOT 1
- Accuracy of XGBClassifier of disable mode HOT 2
- GPU acceleration in concrete-ml ? HOT 3
- Making a fast satellite image classification use case [was: Some questions about GPU acceleration] HOT 24
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from concrete-ml.