Giter VIP home page Giter VIP logo

giskard-ai / giskard Goto Github PK

View Code? Open in Web Editor NEW
3.1K 3.1K 199.0 180.05 MB

🐒 Open-Source Evaluation & Testing framework for LLMs and ML models

Home Page: https://docs.giskard.ai

License: Apache License 2.0

Shell 0.04% JavaScript 1.79% HTML 0.87% CSS 0.51% Python 96.80%
ai-red-team ai-safety ai-security ai-testing ethical-artificial-intelligence fairness-ai llm llm-eval llm-evaluation llm-security llmops ml-safety ml-testing ml-validation mlops model-monitoring rag-evaluation red-team-tools responsible-ai trustworthy-ai

giskard's People

Contributors

abssent avatar adrimarteau avatar alexcombessie avatar andreybavt avatar asselindebeauville avatar cy-moi avatar dependabot[bot] avatar googleton avatar goyalpramod avatar hartorn avatar henchaves avatar inokinoki avatar jmsquare avatar kevinmessiaen avatar kranium2002 avatar luca-martial avatar luca-rossi avatar mackerac avatar mathieuroques avatar mattbit avatar neyrii avatar p4puniya avatar pierlj avatar princyiakov avatar qgk75 avatar rabah-khalek avatar smixi avatar stevenbrum avatar teoale avatar vjern avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

giskard's Issues

Error in upload_model_and_df when target value is not declared in column_types

πŸ› Bug Report

When I am trying to run upload_model_and_df function, I get an error : KeyError: 'default' since I did not declare target value in column_types . But this behavior was supported in the previous version.

πŸ”¬ How To Reproduce

Steps to reproduce the behavior:

  1. Select Giskard Credit Scoring Demo Notebook

  2. Remove 'default' column from column_types
    column_types = {'account_check_status':"category",
    'duration_in_month':"numeric",
    'credit_history':"category",
    'purpose':"category",
    'credit_amount':"numeric",
    'savings':"category",
    'present_employment_since':"category",
    'installment_as_income_perc':"numeric",
    'sex':"category",
    'personal_status':"category",
    'other_debtors':"category",
    'present_residence_since':"numeric",
    'property':"category",
    'age':"numeric",
    'other_installment_plans':"category",
    'housing':"category",
    'credits_this_bank':"numeric",
    'job':"category",
    'people_under_maintenance':"numeric",
    'telephone':"category",
    'foreign_worker':"category"}

  3. Run the Notebook after updating your Token

Environment

  • OS: macOS
  • Python version, get it with: 3.9.7
  • giskard PYPI version : 1.7.0
  • giskard app version : 1.3.0
    --

### Screenshots

πŸ“ˆ Expected behavior : Should be able to upload model and data without declaring target in column_types like in the previous version

πŸ“Ž Stack Trace

/Users/princyiakov/opt/anaconda3/lib/python3.9/site-packages/giskard/client/project.py:634: UserWarning: Feature 'people_under_maintenance' is declared as 'numeric' but has 2 (<= nuniques_category=2) distinct values. Are you sure it is not a 'category' feature?
  warning(
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/var/folders/kp/fq5v4zt954n8wtmwm0n_n0j80000gn/T/ipykernel_60553/2045684294.py in <module>
----> 1 credit_scoring.upload_model_and_df(
      2     prediction_function=clf_logistic_regression.predict_proba, # Python function which takes pandas dataframe as input and returns probabilities for classification model OR returns predictions for regression model
      3     model_type='classification', # "classification" for classification model OR "regression" for regression model
      4     df=test_data, # the dataset you want to use to inspect your model
      5     column_types=column_types, # A dictionary with columns names of df as key and types(category, numeric, text) of columns as values

~/opt/anaconda3/lib/python3.9/site-packages/giskard/client/project.py in upload_model_and_df(self, prediction_function, model_type, df, column_types, feature_names, target, model_name, dataset_name, classification_threshold, classification_labels)
    388         """
    389         self.analytics.track("Upload model and dataset")
--> 390         data, raw_column_types = self._validate_and_compress_data(column_types, df, target)
    391         classification_labels, model = self._validate_model(
    392             classification_labels,

~/opt/anaconda3/lib/python3.9/site-packages/giskard/client/project.py in _validate_and_compress_data(self, column_types, df, target)
    329         self.validate_columns_columntypes(df, column_types, target)
    330         self._validate_column_types(column_types)
--> 331         self._validate_column_categorization(df, column_types)
    332         raw_column_types = df.dtypes.apply(lambda x: x.name).to_dict()
    333         data = compress(save_df(df))

~/opt/anaconda3/lib/python3.9/site-packages/giskard/client/project.py in _validate_column_categorization(df, feature_types)
    630         for column in df.columns:
    631             if nuniques[column] <= nuniques_category and \
--> 632                     (feature_types[column] == SupportedColumnType.NUMERIC.value or \
    633                      feature_types[column] == SupportedColumnType.TEXT.value):
    634                 warning(

KeyError: 'default'

TFAutoModel has issues while unpickling

GSK-285

TFAutomodel does not have the .bert function of TFBertModel which solves the known unpicking issue (huggingface/transformers#3627) forcing the users to use only TFBertModel instead of TFAutomodel. This is NOT OUR BUG but we will face integration issues in future. Creating card to keep track of the issue.

Project: Giskard
Reporter: princy
Created: 2022-09-16

Make LIME categories searcheable

GSK-394

Max from Unifai:

Ah et autre feedback d’UI <@U02GW359JDV>: quand je veux tester un autre label pour le lime, par exemple le actual au lieu du predicted, et que j’ai beaucoup de labels, je suis obligΓ© de scroller pour le trouver.

Γ§a serait pas mal une petite search, lΓ  j’en ai 697, et pour trouver le bon code c’est rock’n’roll

{width=70%}

Project: Giskard
Reporter: aavtomonov
Created: 2022-12-02

get rid of Lime html generation

GSK-355

currently we call text_explainer.show_prediction to generate Lime explanations. It requires us to install ipython just to be able to generate HTML on top of raw Lime values.

Instead of adding ipython dependency we should visualize lime values on the frontend

Project: Giskard
Reporter: aavtomonov
Created: 2022-11-14

Triggered an assert when reduced dataset of ENRON

GSK-362

When we reduced the datasets of ENRON down to 10, it triggered the following assert:

            assert not len(
                non_declared_categories
            ), f'Predicted classes don\'t exist in the dataset "{actual_slice.target}" column: {non_declared_categories}'

giving:

  File "/Users/rak/Documents/giskard-client/giskard/ml_worker/testing/performance_tests.py", line 61, in test_auc
    ), f'Predicted classes don\'t exist in the dataset "{actual_slice.target}" column: {non_declared_categories}'
AssertionError: Predicted classes don't exist in the dataset "Target" column: {'INTERNAL'}

that's because the reduced dataset's target didn't cover all the possible outputs that exist in the full dataset, and more specifically that are predicted by the model.

This causes only the AUC test to fail.

Best solution: we should be more careful in the way we reduce the dataset and redo the exercise by picking entries that have all the possible labels.

linked to https://giskard.youtrack.cloud/issue/GSK-342/Make-all-the-tests-that-are-automatically-generated-from-demo-projects-work

Project: Giskard
Reporter: rabah
Created: 2022-11-15

Auto-generated project key doesn't meet validation

πŸ› Bug Report

The automatically generated project key replaces spaces with dashes. The project key validation doesn't allow the field to have dashes.

πŸ”¬ How To Reproduce

Steps to reproduce the behavior:

  1. Open the dashboard and log in with your account;
  2. Go to Projects;
  3. Create a new project;
  4. Type a project name containing spaces;
  5. Don't edit the auto-generated project key;
  6. Click on SAVE.

Screenshots

Screen Shot 2022-06-23 at 15 32 35

No 'good first issues'.

No Good First Issues.

As a budding community, it's necessary to attract the new talent towards one's organisation. And 'G.F.I.' is the best practice to do that. But I didn't find any such issues.

πŸ“Ž Additional context

A few good first issues can be:

  • Modifying the documentations.
  • Updating tags such as version, Licence etc in the readme.md and other documents whenever there's a new update.
  • Adding hyperlinks to the necessary models (such as PyTorch, TensorFlow etc) to make the docs more interactive.
  • Adding operating demos about a particular feature.
  • Other issues that are time consuming but simple and easy.
    As long as there's something for the newcomers to do... The community will grow.

Run all tests fail on sentiment analysis model for Python 3.7 on mac

GSK-364

Setup: Giskard installed on SoYouStart: http://ns3031812.ip-188-165-228.eu:19000/main/projects/42/test-suites/49/

ML worker running on macOS Python 3.7 with Rosetta

Here are the ML worker logs:

2022-11-15 22:47:00,218 pid:40900 MainThread giskard.cli  INFO     Starting ML Worker client
2022-11-15 22:47:00,218 pid:40900 MainThread giskard.cli  INFO     Python: /Users/alexcombessie/Github/giskard-examples/venv/bin/python3.7 (3.7.13)
2022-11-15 22:47:00,219 pid:40900 MainThread giskard.cli  INFO     Giskard Home: /Users/alexcombessie/giskard-home
2022-11-15 22:47:02,364 pid:40900 MainThread giskard.ml_worker.ml_worker INFO     Started ML Worker server on port 62633
2022-11-15 22:47:02,365 pid:40900 MainThread giskard.ml_worker.ml_worker INFO     Remote server host and port are specified, connecting as an external ML Worker
2022-11-15 22:47:02,365 pid:40900 MainThread giskard.ml_worker.bridge.ml_worker_bridge INFO     Connecting to ns3031812.ip-188-165-228.eu:40051
2022-11-15 22:47:02,420 pid:40900 MainThread giskard.ml_worker.bridge.ml_worker_bridge INFO     Connected to Giskard server ns3031812.ip-188-165-228.eu:40051
2022-11-15 22:47:07,484 pid:40900 ThreadPoolExecutor-0_2 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:07,501 pid:40900 ThreadPoolExecutor-0_3 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:07,503 pid:40900 ThreadPoolExecutor-0_0 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:07,517 pid:40900 ThreadPoolExecutor-0_7 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:07,529 pid:40900 ThreadPoolExecutor-0_1 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:07,544 pid:40900 ThreadPoolExecutor-0_11 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:07,545 pid:40900 ThreadPoolExecutor-0_12 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:07,561 pid:40900 ThreadPoolExecutor-0_9 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:07,561 pid:40900 ThreadPoolExecutor-0_21 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:07,577 pid:40900 ThreadPoolExecutor-0_5 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:07,838 pid:40900 ThreadPoolExecutor-0_24 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.261245
2022-11-15 22:47:07,840 pid:40900 ThreadPoolExecutor-0_0 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.279598
2022-11-15 22:47:07,843 pid:40900 ThreadPoolExecutor-0_2 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.308509
2022-11-15 22:47:07,843 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:07,845 pid:40900 ThreadPoolExecutor-0_6 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.310792
2022-11-15 22:47:07,847 pid:40900 ThreadPoolExecutor-0_16 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.253726
2022-11-15 22:47:07,849 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:07,850 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:07,851 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:07,851 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:07,903 pid:40900 ThreadPoolExecutor-0_20 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:07,917 pid:40900 ThreadPoolExecutor-0_21 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:07,918 pid:40900 ThreadPoolExecutor-0_10 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:07,934 pid:40900 ThreadPoolExecutor-0_24 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:07,934 pid:40900 ThreadPoolExecutor-0_2 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:07,954 pid:40900 ThreadPoolExecutor-0_39 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.019753
2022-11-15 22:47:07,954 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:07,957 pid:40900 ThreadPoolExecutor-0_26 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:07,957 pid:40900 ThreadPoolExecutor-0_17 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:07,976 pid:40900 ThreadPoolExecutor-0_18 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.020004
2022-11-15 22:47:07,977 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:07,979 pid:40900 ThreadPoolExecutor-0_28 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:07,999 pid:40900 ThreadPoolExecutor-0_8 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.020348
2022-11-15 22:47:08,000 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:08,020 pid:40900 ThreadPoolExecutor-0_14 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.019028
2022-11-15 22:47:08,020 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:08,022 pid:40900 ThreadPoolExecutor-0_20 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:08,037 pid:40900 ThreadPoolExecutor-0_33 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:08,045 pid:40900 ThreadPoolExecutor-0_10 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:08,069 pid:40900 ThreadPoolExecutor-0_36 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.018241
2022-11-15 22:47:08,069 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:08,071 pid:40900 ThreadPoolExecutor-0_16 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:08,071 pid:40900 ThreadPoolExecutor-0_6 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:08,088 pid:40900 ThreadPoolExecutor-0_24 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:08,088 pid:40900 ThreadPoolExecutor-0_40 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:08,106 pid:40900 ThreadPoolExecutor-0_7 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.018054
2022-11-15 22:47:08,107 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:08,108 pid:40900 ThreadPoolExecutor-0_26 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:08,127 pid:40900 ThreadPoolExecutor-0_18 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.018522
2022-11-15 22:47:08,127 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:08,146 pid:40900 ThreadPoolExecutor-0_45 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.017542
2022-11-15 22:47:08,146 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:08,148 pid:40900 ThreadPoolExecutor-0_11 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:08,162 pid:40900 ThreadPoolExecutor-0_22 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:08,162 pid:40900 ThreadPoolExecutor-0_14 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:08,175 pid:40900 ThreadPoolExecutor-0_9 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:08,177 pid:40900 ThreadPoolExecutor-0_23 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:08,197 pid:40900 ThreadPoolExecutor-0_49 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.020026
2022-11-15 22:47:08,197 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:08,200 pid:40900 ThreadPoolExecutor-0_33 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:08,219 pid:40900 ThreadPoolExecutor-0_21 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.019226
2022-11-15 22:47:08,219 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:08,220 pid:40900 ThreadPoolExecutor-0_35 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:08,239 pid:40900 ThreadPoolExecutor-0_36 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.018733
2022-11-15 22:47:08,240 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:08,241 pid:40900 ThreadPoolExecutor-0_37 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:08,253 pid:40900 ThreadPoolExecutor-0_41 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:08,275 pid:40900 ThreadPoolExecutor-0_42 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.019104
2022-11-15 22:47:08,275 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:08,277 pid:40900 ThreadPoolExecutor-0_26 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:08,277 pid:40900 ThreadPoolExecutor-0_18 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:08,290 pid:40900 ThreadPoolExecutor-0_44 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:08,309 pid:40900 ThreadPoolExecutor-0_8 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.018262
2022-11-15 22:47:08,309 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:08,311 pid:40900 ThreadPoolExecutor-0_11 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:08,331 pid:40900 ThreadPoolExecutor-0_30 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.020889
2022-11-15 22:47:08,332 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:08,333 pid:40900 ThreadPoolExecutor-0_47 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:08,346 pid:40900 ThreadPoolExecutor-0_22 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:08,365 pid:40900 ThreadPoolExecutor-0_20 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.018886
2022-11-15 22:47:08,365 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:08,366 pid:40900 ThreadPoolExecutor-0_9 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:08,397 pid:40900 ThreadPoolExecutor-0_23 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.017651
2022-11-15 22:47:08,397 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:16,928 pid:40900 ThreadPoolExecutor-0_25 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:16,945 pid:40900 ThreadPoolExecutor-0_38 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:16,949 pid:40900 ThreadPoolExecutor-0_24 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:16,961 pid:40900 ThreadPoolExecutor-0_7 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:16,966 pid:40900 ThreadPoolExecutor-0_19 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:16,995 pid:40900 ThreadPoolExecutor-0_40 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.027576
2022-11-15 22:47:16,995 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:16,997 pid:40900 ThreadPoolExecutor-0_29 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:16,998 pid:40900 ThreadPoolExecutor-0_43 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:17,019 pid:40900 ThreadPoolExecutor-0_45 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.020985
2022-11-15 22:47:17,019 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:17,021 pid:40900 ThreadPoolExecutor-0_12 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:17,040 pid:40900 ThreadPoolExecutor-0_11 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.019098
2022-11-15 22:47:17,040 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:17,042 pid:40900 ThreadPoolExecutor-0_4 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:17,061 pid:40900 ThreadPoolExecutor-0_14 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.019765
2022-11-15 22:47:17,062 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:17,064 pid:40900 ThreadPoolExecutor-0_15 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:17,078 pid:40900 ThreadPoolExecutor-0_34 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:17,099 pid:40900 ThreadPoolExecutor-0_3 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.018919
2022-11-15 22:47:17,100 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:17,101 pid:40900 ThreadPoolExecutor-0_36 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:17,101 pid:40900 ThreadPoolExecutor-0_25 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:17,116 pid:40900 ThreadPoolExecutor-0_16 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:17,135 pid:40900 ThreadPoolExecutor-0_41 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.018976
2022-11-15 22:47:17,135 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:17,137 pid:40900 ThreadPoolExecutor-0_27 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:17,156 pid:40900 ThreadPoolExecutor-0_17 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.019178
2022-11-15 22:47:17,156 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/grpc_mapper.py", line 23, in deserialize_model
    ZstdDecompressor().stream_reader(model_stream)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 649, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'torch.tensor'
2022-11-15 22:47:17,158 pid:40900 ThreadPoolExecutor-0_19 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/datasets/data_43.zst
2022-11-15 22:47:17,172 pid:40900 ThreadPoolExecutor-0_43 giskard.ml_worker.server.ml_worker_service INFO     File already exists: /Users/alexcombessie/giskard-home/projects/sentiment_analysis/models/model_44.zst
2022-11-15 22:47:17,190 pid:40900 ThreadPoolExecutor-0_45 giskard.ml_worker.utils.logging INFO     giskard.ml_worker.utils.grpc_mapper.deserialize_model executed in 0:00:00.018524
2022-11-15 22:47:17,190 pid:40900 MainThread grpc._cython.cygrpc ERROR    Unexpected [ModuleNotFoundError] raised by servicer method [/worker.MLWorker/runTest]
Traceback (most recent call last):
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 682, in grpc._cython.cygrpc._handle_exceptions
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 796, in _handle_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 547, in _handle_unary_unary_rpc
  File "src/python/grpcio/grpc/_cython/_cygrpc/aio/server.pyx.pxi", line 411, in _finish_handler_with_unary_response
  File "/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/server/ml_worker_service.py", line 136, in runTest
    model = deserialize_model(request.model)
  File "/Users/alexcombessie/Github/giskard-examples/venv/lib/python3.7/site-packages/giskard/ml_worker/utils/logging.py", line 74, 

remove feature_names in the upload of model

GSK-390

Problem with unifai, column_names has variable

Feedback by unifai

'fatearesnames" is redudundant with "feature names"

"people forget to put metadata column in column-names"

Project: Giskard
Reporter: jmjm
Created: 2022-11-29

H2O models unable to unpickle on Giskard

πŸ› Bug Report

The bug occurs during _validate_model_is_pickleable of H2OAutoML:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/Documents/giskard-client/.venv/lib/python3.7/site-packages/giskard/client/project.py in _validate_model_is_pickleable(prediction_function)
    674             pickled_model = cloudpickle.dumps(prediction_function)
--> 675             unpickled_model = cloudpickle.loads(pickled_model)
    676         except Exception:

TypeError: __new__() missing 1 required positional argument: 'keyvals'

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-55-c024ce4c5c6b> in <module>
     34     classification_labels=['Yes','No'],
     35     model_name='test_model',
---> 36     dataset_name='test_data'
     37     )

~/Documents/giskard-client/.venv/lib/python3.7/site-packages/giskard/client/project.py in upload_model_and_df(self, prediction_function, model_type, df, column_types, feature_names, target, model_name, dataset_name, classification_threshold, classification_labels)
    396             prediction_function,
    397             target,
--> 398             df,
    399         )
    400         data_res = self._post_data(column_types, data, dataset_name, raw_column_types, target)

~/Documents/giskard-client/.venv/lib/python3.7/site-packages/giskard/client/project.py in _validate_model(self, classification_labels, classification_threshold, feature_names, model_type, prediction_function, target, validate_df)
    222             validate_df,
    223     ):
--> 224         prediction_function = self._validate_model_is_pickleable(prediction_function)
    225         transformed_pred_func = self.transform_prediction_function(
    226             prediction_function, feature_names

~/Documents/giskard-client/.venv/lib/python3.7/site-packages/giskard/client/project.py in _validate_model_is_pickleable(prediction_function)
    675             unpickled_model = cloudpickle.loads(pickled_model)
    676         except Exception:
--> 677             raise ValueError("Unable to pickle or unpickle model on Giskard")
    678         return unpickled_model
    679 

ValueError: Unable to pickle or unpickle model on Giskard

πŸ”¬ How To Reproduce

Run the following script: -- (TBD)

Environment

  • Environment independent problem, but tried on: Windows / macOS.
  • Python version independent problem, but tried with both 3.7 and 3.10

πŸ“Ž Additional context

Similar mentions by other people too (like https://groups.google.com/g/h2ostream/c/FaJm57Rbkmg?pli=1)
One of the proposed solution (to be tested) is to:

  • use the internal save_model function in H2O,
  • load it in to python as a binary object,
  • pickle the binary instead.

Error in Explanation of a pytorch NLP model

GSK-393

I'm currently getting the following error when attempting to explain a NLP model:

Traceback (most recent call last):
  File "/Users/rak/Documents/giskard/python-client/giskard/ml_worker/core/model_explanation.py", line 67, in explain_text
    text_explainer.fit(text_document, prediction_function)
  File "/Users/rak/Documents/giskard/python-client/.venv/lib/python3.7/site-packages/eli5/lime/lime.py", line 265, in fit
    random_state=self.rng_
  File "/Users/rak/Documents/giskard/python-client/.venv/lib/python3.7/site-packages/eli5/lime/lime.py", line 367, in _train_local_classifier
    random_state=rng)
  File "/Users/rak/Documents/giskard/python-client/.venv/lib/python3.7/site-packages/eli5/lime/utils.py", line 32, in fit_proba
    random_state=random_state,
  File "/Users/rak/Documents/giskard/python-client/.venv/lib/python3.7/site-packages/eli5/lime/utils.py", line 111, in expanded_X_y_sample_weights
    sample_weight
  File "/Users/rak/Documents/giskard/python-client/.venv/lib/python3.7/site-packages/eli5/lime/utils.py", line 145, in expand_dataset
    for label in rng.choice(classes, size=factor, p=probs):
  File "mtrand.pyx", line 937, in numpy.random.mtrand.RandomState.choice
ValueError: probabilities are not non-negative

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/rak/Documents/giskard/python-client/giskard/ml_worker/utils/error_interceptor.py", line 43, in wrapper
    res = behavior(request, context)
  File "/Users/rak/Documents/giskard/python-client/giskard/ml_worker/server/ml_worker_service.py", line 182, in explainText
    html_response = explain_text(model, input_df, text_column, text_document, n_samples)._repr_html_()
  File "/Users/rak/Documents/giskard/python-client/giskard/ml_worker/utils/logging.py", line 74, in wrap
    result = fn(*args, **kw)
  File "/Users/rak/Documents/giskard/python-client/giskard/ml_worker/core/model_explanation.py", line 71, in explain_text
    raise Exception("Failed to create text explanation") from e
Exception: Failed to create text explanation

Project: Giskard
Reporter: rabah
Created: 2022-11-30

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.