How to compose hummingbird model with other torchscript models about hummingbird HOT 4 CLOSED

Hi @jamespinkerton, Can you please post the error here?

We provide the save/load functions which work to serialize any model (including torchscript), but if I understand correctly this won't work for your use case as you need to deserialize it in a non-python environment? I wonder if any of that save/load torch jit code would be at all useful in some other way (the logic translated to some other language)?

from hummingbird.

To add to what Karla said, if you are calling torch.jit.script on an Hummingbird model it won't work. But we provide already that functionality for you: you can compile your model by passing torch.jit as backed and then save it as Karla said and you should be ok (assuming you manually deserialize the model on the C++ side following our loading code as Karla already suggested). Example on how to generate torchscript models from Hummingbird are here.

from hummingbird.

This is a great answer, but I don't think it quite answers my question. When I use a modeling package to solve a real-world problem, I usually need to surround it with other things. In this case, I surround it with a torch model. Think of that as preprocessing and postprocessing that occur before and after LGBM.

So fundamentally I need to be able to compose operations. I also need to be able to move the model into torchscript. Based on your answer, I don't see how I could do both at once. If I compose pytorch and hummingbird I can't later move to torchscript. If I compile hummingbird to torchscript first, I don't see how to compose it with other things in pytorch (which themselves have to be put in torchscript too).

from hummingbird.

Ok got it. Have you tried with torch.jit.trace instead of script? That is what we are using internally so maybe it will also work in your case?

from hummingbird.