Comments (6)
shayaks
commented on June 15, 2024
1
Ah you're right, missed that fact that this is the pytorch notebook. Will take a look and get back soon. Thanks for trying this out!
from trulens.
shayaks
commented on June 15, 2024
Hey Ashish, this error is because of an issue with Tensorflow 2.6 interfaces. #46 should fix this. In the meanwhile, feel free to try Tensorflow 2.4 or 2.5.
from trulens.
AshishSardana
commented on June 15, 2024
Hi Shayak, this error originates from the PyTorch backend. Should this have any correlation with TF version?
from trulens.
klasleino
commented on June 15, 2024
Hi Ashish,
I think the issue is that the output to the model is an OrderedDict rather than a tensor. TruLens can still work with models with other types of outputs, but you will need a custom quantity of interest (QoI) to handle them.
If you can provide a bit more information about the output of your segmentation model, I can maybe give some advice on how to create your quantity of interest.
from trulens.
AshishSardana
commented on June 15, 2024
Klas, I'm using the segmentation model from Torchvision i.e. torchvision.models.segmentation.fcn_resnet50
The layer's of the model are attached below. I'm not specifying the from/to layer slices while initializing the InputAttributions class, hence I suppose it'll automatically consider the first and last layers as slices.
Its a pretty standard segmentation model. Let me know if you'd like me to share more details.
'backbone_bn1': BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_relu': ReLU(inplace=True)
'backbone_maxpool': MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
'backbone_layer1_0_conv1': Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer1_0_bn1': BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer1_0_conv2': Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
'backbone_layer1_0_bn2': BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer1_0_conv3': Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer1_0_bn3': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer1_0_relu': ReLU(inplace=True)
'backbone_layer1_0_downsample_0': Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer1_0_downsample_1': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer1_1_conv1': Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer1_1_bn1': BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer1_1_conv2': Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
'backbone_layer1_1_bn2': BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer1_1_conv3': Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer1_1_bn3': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer1_1_relu': ReLU(inplace=True)
'backbone_layer1_2_conv1': Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer1_2_bn1': BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer1_2_conv2': Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
'backbone_layer1_2_bn2': BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer1_2_conv3': Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer1_2_bn3': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer1_2_relu': ReLU(inplace=True)
'backbone_layer2_0_conv1': Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer2_0_bn1': BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer2_0_conv2': Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
'backbone_layer2_0_bn2': BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer2_0_conv3': Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer2_0_bn3': BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer2_0_relu': ReLU(inplace=True)
'backbone_layer2_0_downsample_0': Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
'backbone_layer2_0_downsample_1': BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer2_1_conv1': Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer2_1_bn1': BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer2_1_conv2': Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
'backbone_layer2_1_bn2': BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer2_1_conv3': Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer2_1_bn3': BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer2_1_relu': ReLU(inplace=True)
'backbone_layer2_2_conv1': Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer2_2_bn1': BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer2_2_conv2': Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
'backbone_layer2_2_bn2': BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer2_2_conv3': Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer2_2_bn3': BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer2_2_relu': ReLU(inplace=True)
'backbone_layer2_3_conv1': Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer2_3_bn1': BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer2_3_conv2': Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
'backbone_layer2_3_bn2': BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer2_3_conv3': Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer2_3_bn3': BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer2_3_relu': ReLU(inplace=True)
'backbone_layer3_0_conv1': Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer3_0_bn1': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_0_conv2': Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
'backbone_layer3_0_bn2': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_0_conv3': Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer3_0_bn3': BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_0_relu': ReLU(inplace=True)
'backbone_layer3_0_downsample_0': Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer3_0_downsample_1': BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_1_conv1': Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer3_1_bn1': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_1_conv2': Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
'backbone_layer3_1_bn2': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_1_conv3': Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer3_1_bn3': BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_1_relu': ReLU(inplace=True)
'backbone_layer3_2_conv1': Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer3_2_bn1': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_2_conv2': Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
'backbone_layer3_2_bn2': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_2_conv3': Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer3_2_bn3': BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_2_relu': ReLU(inplace=True)
'backbone_layer3_3_conv1': Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer3_3_bn1': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_3_conv2': Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
'backbone_layer3_3_bn2': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_3_conv3': Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer3_3_bn3': BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_3_relu': ReLU(inplace=True)
'backbone_layer3_4_conv1': Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer3_4_bn1': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_4_conv2': Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
'backbone_layer3_4_bn2': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_4_conv3': Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer3_4_bn3': BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_4_relu': ReLU(inplace=True)
'backbone_layer3_5_conv1': Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer3_5_bn1': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_5_conv2': Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
'backbone_layer3_5_bn2': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_5_conv3': Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer3_5_bn3': BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer3_5_relu': ReLU(inplace=True)
'backbone_layer4_0_conv1': Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer4_0_bn1': BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer4_0_conv2': Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
'backbone_layer4_0_bn2': BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer4_0_conv3': Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer4_0_bn3': BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer4_0_relu': ReLU(inplace=True)
'backbone_layer4_0_downsample_0': Conv2d(1024, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer4_0_downsample_1': BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer4_1_conv1': Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer4_1_bn1': BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer4_1_conv2': Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(4, 4), dilation=(4, 4), bias=False)
'backbone_layer4_1_bn2': BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer4_1_conv3': Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer4_1_bn3': BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer4_1_relu': ReLU(inplace=True)
'backbone_layer4_2_conv1': Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer4_2_bn1': BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer4_2_conv2': Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(4, 4), dilation=(4, 4), bias=False)
'backbone_layer4_2_bn2': BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer4_2_conv3': Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
'backbone_layer4_2_bn3': BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'backbone_layer4_2_relu': ReLU(inplace=True)
'classifier_0': Conv2d(2048, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
'classifier_1': BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'classifier_2': ReLU()
'classifier_3': Dropout(p=0.1, inplace=False)
'classifier_4': Conv2d(512, 21, kernel_size=(1, 1), stride=(1, 1))
'aux_classifier_0': Conv2d(1024, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
'aux_classifier_1': BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
'aux_classifier_2': ReLU()
'aux_classifier_3': Dropout(p=0.1, inplace=False)
'aux_classifier_4': Conv2d(256, 21, kernel_size=(1, 1), stride=(1, 1))
from trulens.
klasleino
commented on June 15, 2024
Sorry, I meant the output of the model, not the architecture. I'm not particularly familiar with torchvision, but looking into it, it seems that the output of torchvision models is an OrderedDict and not a torch.Tensor, which by default has a single key, 'out'. If this is the case for your model as well, you could try the following fix:
class TorchvisionMaxClassQoI(MaxClassQoI):
def __call__(self, y):
super().__call__(y['out']) Then when you create your attribution measure you can use this QoI:
infl = InputAttribution(model, qoi=TorchvisionMaxClassQoI())
attrs_input = infl.attributions(x_pp)This assumes the output of your model is a dictionary containing a key, 'out', representing the model's actual output. If the output is different from that, you'll want to adjust how you extract the output tensor accordingly.
from trulens.
Related Issues (20)
- Import Error from Optional Requirements (LiteLLM) HOT 2
- Migrate Llama-Index Optional Dependency to 0.10 HOT 1
- Tru() fails with `database_url` key HOT 2
- Error Executing 'llama_index_quickstart.py' and Integrating with RAG: Common Issue Across Examples. HOT 8
- Error - OpenAPI request failed
- RuntimeError: cannot reuse already awaited coroutine HOT 4
- Error Executing llama_index_quickstart.ipynb HOT 12
- Runtime Error on Langchain Quickstart HOT 2
- Llama Index quick start - str object has no attribute query_str HOT 4
- tru.run_dashboard error HOT 2
- Found more than one `BaseRetriever` in app HOT 2
- Could not locate app.get_relevant_documents.rets[:].page_content in app/record. HOT 2
- requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) HOT 3
- Evaluation metrics inference HOT 3
- [BUG]: When attribute is in String instead of response format , it gives error:
- [BUG] How do I evaluate my Llama-Index app for hallucination? HOT 1
- [FEAT] Adding support of base feedback functions to different languages HOT 2
- RuntimeError: Could not locate app.query.rets.source_nodes[:].note.text in app/record. HOT 5
- how can I use LLM from huggingface as the provider of the Trulens feedback? HOT 4
- [BUG] ValidationError: 1 validation error for DataSource component_type subclass of BaseComponent expected (type=type_error.subclass; expected_class=BaseComponent) HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from trulens.