Client-Server FHE with a non-concrete_ml torch model about concrete-ml HOT 7 CLOSED

Hi, thanks again for your quick response!

In the end I decided to use QAT as you recommended and leave the other approach aside. I found the step-by-step guide very useful!!.
I therefore proceed to close the issue.

Thank you very much!!

from concrete-ml.

Hello again,

Thanks a lot for the new infos ! Just to make sure, did you apply any transformations on your inputs since you have trained your model ? For example, I can see that your inputs are flattened, but was that already the case when your model was being trained ? Be aware that inputs' shape should remain the same between fitting, compilation and prediction steps !

Besides, what error did you get when compiling with 6 bits ? If it is something like Cannot find crypto parameters, this is most likely expected.

In fact, there's a few things to tell here. First, Concrete ML 0.6 does indeed handle 16 bits of precision, but setting n_bits=6 will most likely make your model exceed this constraint, especially with complex structures such as fully-connected neural networks. Such an overflow will make your model not being able to compile. If you want to know more about it, you can check our documentation about quantization, Neural Networks and our use of pruning !

Then, you might also want to convert your model to a Quantize Aware Training (QAT) one if you haven't done it yet ! The main reason for this is that neural networks perform much better under quantization constraints with such a technique. Besides, this conversion can easily be done using the Brevitas library ! You'll find more information in our step_by_step guide and how to import QAT models.

Finally, I invite you to check some of our examples if you haven't done it yet ! Among them, here's how we tackled the MNIST classification problem with our own built-in FC neural networks (which use QAT), reaching 97% accuracy with a few bits of quantization. We also give an example of how to solve this task with a custom model in one of our use case examples. If you are not able to train your model again using QAT, here's a more advanced example on how we succeeded to convert a pre-trained model to a QAT one and use it in Concrete ML for tackling the CIFAR datasets.

Let me know if you have more questions !

from concrete-ml.

Hello !

Great to see you experimenting with our Client-Server interface ! Based the error you get, I would say you probably have an issue with your inputs' shape. More precisely, I think your encrypted_input's shape might not match the one used for your torch_inputset's samples that you gave to compile_torch_model.

However, it is a bit hard to know more. Would it be possible to give us access to your code or at least tell us more about what you are trying to achieve so that we can better understand the issue ?

Thanks !

from concrete-ml.

Hi, thank you very much for your quick response!!
I have been reviewing the dimensions, and it seems this is not the problem (even though the error raises that exception) 
Trying some parameter combinations, I have compiled the torch model again but with 2 bits (I had n_bit=6) (see the snippet below).
With this modification, I'm able to reproduce the original client-server tutorial, but the accuracy is very low.
In the last release I though the number of bits was increased to 16 but I don't know why 6 bits did not work but everything seems ok with 2.

Code snippet

from concrete.ml.torch.compile import compile_torch_model
quantized_numpy_module = compile_torch_model(
    torch_model=torch_fc_model, # this is the well-known mnist model, 
    torch_inputset=torch_inputset, # this tensor has the appropiate dimensions  
    n_bits=2)

Moreover, in another attempt to try the client-server tutorial, I've realised that the compile_torch_model function raises an error, depending on the combination of the torch_inputset and the number of bits. I've tried setting as the torch_inputset an array of the MNIST dataset ( torch_inputset = mnist_test_data[:-10]), as well as a random array , both with the same vector dimensions:

input_output_feature = 784
input_shape = (input_output_feature,)
n_examples = 5000
# Create random input
torch_inputset = numpy.random.uniform(-100, 100, size=(n_examples, *input_shape))

Does the configuration of the number of bits depends on the size of the input?
Why does the whole tutorial work with 2 bits, but not with more than two?

Thanks a lot!!

from concrete-ml.

Hello again, thank you very much for your response!

I've been revising my code and I confirmed that my inputs are flattened in training, compilation and prediction, so same shapes are always used in each step.

When I try to use the function compile_torch_model with 6 bits, the error I'm getting is:

RuntimeError

...
in compile_torch_model(torch_model, torch_inputset, import_qat, configuration, compilation_artifacts, show_mlir, n_bits, use_virtual_lib, p_error, global_p_error, verbose_compilation) 

%21 = 0 # ClearScalar<uint1> 
%22 = 63 # ClearScalar<uint6>
%23 = clip(%20, %21, %22) # EncryptedTensor<float64, shape=(1, 64)> 
%24 = astype(%23, dtype=int_) # EncryptedTensor<uint1, shape=(1, 64)> 
return %24

About converting the model to a QAT one, I thought this process was done in the function compile_torch_model since the output is a QuantizedModule. Am I forgetting to do an additional step? Is it preferable to use Brevitas?

Finally, I've tried the MNIST and CIFAR examples and they really get high accuracy so I will rely my next steps on these examples.

Thank you very much!

from concrete-ml.

Hello,
Thanks again for your feedback !

The error you give is not enough to determine the exact issue you're facing. Could you maybe share the full traceback ?

Besides, we only support models that are either already quantized using QAT or "float models" on which we then perform Post-Training Quantization (PTQ). If you call compile_torch_model on your model without setting import_qat to True, the function considers that your model falls into this second category and I believe this is your case. However, PTQ does not seem to work very well with Neural Networks, especially with small numbers of bits. Our built-in Neural Network models use QAT by default for this particular reason. If you want to use use QAT, I therefore invite you to follow our step-by-step guide on how you can indeed use Brevitas to do so !

It is great to know that you have been able to get good results with our examples. Let me know how it goes !

from concrete-ml.

Awesome 😄

If you ever need to ask some further questions or encounter other issues, we are also very active on FHE.org's discord server ("Concrete" channel). Similarly, you can reach out to us on our community forum, where many recurrent questions were already addressed. Don't forget to also star our repository if you find Concrete-ML interesting !

Have a nice day!

from concrete-ml.