'Dream' the best possible interface/combination of visualizations, to best serve our visualization related user stories.
Result could be one or multiple images or a description or both.

Continuous code quality can be handled by Sonarcloud. This repository is configured to use Sonarcloud to perform quality analysis and code coverage report on each push.

In order to configure Sonarcloud analysis GitHub Action workflow you must follow the steps below:

1. go to Sonarcloud to create a new Sonarcloud project
2. login with your GitHub account
3. add Sonarcloud organization or reuse existing one
4. set up a repository
5. go to new code definition administration page and select Number of days option
6. To be able to run the analysis:
   1. a token must be created at Sonarcloud account
   2. the created token must be added as SONAR_TOKEN to secrets on GitHub

And its dependency on captum

3.6 is end of life on 23 Dec 2021, after which it will no longer receive security updates.

https://nlesc.sharepoint.com/:w:/s/DIANNA/EdoppPpmTTBKh0Mq-VdmvM8B_VeTFRzznZ3op-wo1cuBPA?e=qtGO5M

The actual feedback is in our mailboxes.

Link to schematic: https://app.diagrams.net/#Wb!UPZtKeipi0WrFvKBlrEPP4gCzfYL1_ZNqqeyKlYhp43xuB7DeGf3QZ8Mqz5yc_MZ%2F01YA33DUP4W7ISPOFPEBBJTNI4MHVRSL4E

At least on the leafs dataset:

• copy a generated heatmap to this issue
• lesson learnt from implementation
• at least a few lines on the user experience

Focus on the question: Can we reuse (parts of) this implementation?

At least on the leafs dataset:

• copy a generated heatmap to this issue
• lesson learnt from implementation
• at least a few lines on the user experience

Focus on the question: Can we reuse (parts of) this implementation?

3.10 is now a full release and we should support it.

Should include a test that predictions are the same.

In some cases, the user may provide data with one data type, but the XAI methods produces model input with another data type. We should keep track of this, and convert the data back to the correct type before feeding it to the model.

Two links on (test)pypi dianna page are not working (link to "project_setup.md" and "contribution guidelines"). We need to update them.

test_rise_filename uses the MNIST model, but doesn't take into account that it needs a swap of the channels axis. (Model has it as 2nd axis, RISE wants it to be the last axis).
Now the heatmaps produced have a shape of (1, 28) instead of (28, 28). The test passes, because it does match the input shape and that is the only thing we test for.
The test could use the preprocessing_function option to reshape the data before running the test, and perhaps explicitly test if the shape matches the (y, x) shape of the input.

Keep implementing operations and fixing errors until it works. Spend no more than 1 week. Then evaluate with team. Also see #39.

LIME explain_image directly returns
mask = explanation.get_image_and_mask(label, positive_only=False, hide_rest=True, num_features=num_features)[1]
without giving the user the opportunity to change positive_only, hide_rest, or play with parameters such as min_weight.

What was the reasoning behind this choice of default parameters?
Would it be useful to give the user the opportunity to pass those values if needed?

The user will likely stumble upon the base LIME implementation documentation even when trying to use the DIANNA wrapper and that documentation demonstrates interesting flexibility in how to visualize the mask:
https://marcotcr.github.io/lime/tutorials/Tutorial%20-%20images.html

using their CI setup

Currently it needs quite a lof of code to run; most of that can be made part of the model itself.
This would allow for easy use of the model in tests as well

A few times, and again today from Matthieu, the issue came up that getting the tensor axes in the correct order is a huge annoyance and/or confuses users. The preprocessing function provides a way to fix it, but it is still difficult to wrap one's head around.

To make not only DIANNA but really all of tensor-based deep learning more convenient, it would help if we had a tensor type with annotated axes. Something that acts like

class AnnotatedAxesTensor:
    def __init__(self, data, axis_labels):
        ...

batch_size = 20
channels = 38
fidgets = 10
worbles = 204
sigwatts = 2

data = np.zeros((batch_size, channels, fidgets, worbles, sigwatts))

t = AnnotatedAxesTensor(data, ["batch", "channel", "fidget", "worble", "sigwatt"])

print(t.batch[0])

which would output the first batch. Models that use these tensors will be able to "transpose" / rotate axes in the underlying tensor automatically, which will make the user input side a lot easier. It can also give a warning when it does indeed need to reshape the underlying tensor, because this may impact performance, so the user may want to reshape beforehand, but now it will be easy, because the implementation warning can just say exactly how they should do the reshaping:

print("please reshape for optimal performance: t.rotate_axes["batch", "sigwatt", "worble" ...]")

I realize that this may be out of scope for the DIANNA project currently, but maybe this is something to look into in the future, if you think it would be useful.

I have tried using dianna on a basic CLIP pipeline (https://github.com/openai/CLIP).

CLIP takes the output of preprocess(img) as input, where preprocess is model-specific (but provided) and img is a PIL.Image. Once in that format, I can simply do model.encode_image(input)

My goal was therefore to run something like:

    explainer = RISE()
    heatmaps = explainer.explain_image(run_model, x)

Where run_model would include a typical CLIP call (i.e., that function can be used independently from a CLIP+dianna pipeline). What I ended up having was:

    explainer = RISE(preprocess_function=lambda x:np.moveaxis(x, -1, 1))
    x = np.expand_dims(np.moveaxis(images[idx].numpy(), 0, -1), axis=0).astype('double')
    heatmaps = explainer.explain_image(run_model, x, batch_size=50)

Where images[idx] would have been the usual input to CLIP.

Is that the right way to proceed in this situation? Did I miss a simpler approach?

For reference, here is my run_model function:

def run_model(x, predict=False):
    image_input = torch.tensor(np.stack(x)).cuda()

    with torch.no_grad():
        image_features = model.encode_image(image_input).float()
        image_features /= image_features.norm(dim=-1, keepdim=True)

    text_probs = (100.0 * image_features @ text_features.T).softmax(dim=-1)
    top_probs, top_labels = text_probs.topk(5, dim=-1)
    if predict:
        return top_probs, top_labels
    return text_probs.cpu().detach().numpy()

and preprocess is a function similar to

Compose(
    Resize(size=224, interpolation=PIL.Image.BICUBIC)
    CenterCrop(size=(224, 224))
    <function _convert_image_to_rgb at 0x14c2c470e8b0>
    ToTensor()
    Normalize(mean=(0.48145466, 0.4578275, 0.40821073), std=(0.26862954, 0.26130258, 0.27577711))
)

blocked by #56

See if the model can be found in their paper https://thescipub.com/abstract/10.3844/jcssp.2021.349.363
Otherwise we can try emailing the authors asking for their model.

NB. Alternatively, we can use the same pretrained model as they did; replicate the results in their paper. But we should discuss first if we think that's worth it.

It is likely that your CITATION.cff currently doesn't pass validation. The error messages you get from the cffconvert GitHub Action are unfortunately a bit cryptic, but doing the following helps:

• Check if the given-name and family-name keys need updating. If your family name has a name particle like von or van or de, use the name-particle key; if your name has a suffix like Sr or IV, use name-suffix. For details, refer to the schema description: https://github.com/citation-file-format/citation-file-format
• Update the value of the orcid key. If you do not have an orcid yet, you can get one here https://orcid.org/.
• Add more authors if needed
• Update date-released using the YYYY-MM-DD format.
• Update the doi key with the conceptDOI for your repository (see https://help.zenodo.org for more information on what a conceptDOI is). If your project doesn't have a DOI yet, you can use the string 10.0000/FIXME to pass validation.
• Verify that the keywords array accurately describes your project.

Once you do all the steps above, the cffconvert workflow will tell you what content it expected to see in .zenodo.json. Copy-paste from the GitHub Action log into a new file .zenodo.json. Afterwards, the cffconvert GitHub Action should be green.

To help you keep the citation metadata up to date and synchronized, the cffconvert GitHub Action checks the following 6 aspects:

1. Whether your repository includes a CITATION.cff file.

   By including this file, authors of the software can receive credit for the work they put in.

2. Whether your CITATION.cff is valid YAML.

   Visit http://www.yamllint.com/ to see if the contents of your CITATION.cff are valid YAML.

3. Whether your CITATION.cff adheres to the schema (as listed in the CITATION.cff file itself under key cff-version).

   The Citation File Format schema can be found here, along with an explanation of all the keys. You're advised to use the latest available schema version.

4. Whether your repository includes a .zenodo.json file.

   With this file, you can control what metadata should be associated with any future releases of your software on Zenodo: things like the author names, along with their affiliations and their ORCIDs, the license under which the software has been released, as well as the name of your software and a short description. If your repository doesn't have a .zenodo.json file, Zenodo will take a somewhat crude guess to assign these metadata.

   The cffconvert GitHub action will tell you what it expects to find in .zenodo.json, just copy and paste it to a new file named .zenodo.json. The suggested text ignores CITATION.cff's version, commit, and date-released. cffconvert considers these keys suspect in the sense that they are often out of date, and there is little purpose to telling Zenodo about these properties: Zenodo already knows.

5. Whether .zenodo.json is valid JSON.

   Currently unimplemented, but you can check for yourself on https://jsonlint.com/.

6. Whether CITATION.cff and .zenodo.json contain equivalent data.

   This final check verifies that the two files are in sync. The check ignores CITATION.cff's version, commit, and date-released.

All other method tests compare the complete explanation result to a stored one except the RISE test on images, which only compares output shapes.

Standup label: discuss whether to do in this sprint or later.

The deeplift code currently accepts models in pytorch format.
When #39 is done, this should be updated to accept onnx and do the conversion to pytorch internally.

Use results from dianna-ai/dianna-exploration#78.

Create text visualization: ordered table with weights

As a <developer/domain scientist/ML engineer/...> I want in order to .

Link to the list of user stories: https://nlesc.sharepoint.com/:w:/s/DIANNA/EdoppPpmTTBKh0Mq-VdmvM8B_VeTFRzznZ3op-wo1cuBPA?e=qtGO5M

We think that std of the probability of the predicted correct class on all masked instances of an image should be somewhere between 0.2 and 0.4. Under 0.1 we didn't see any good looking heatmaps. Between 0.1 and 0.2 we often saw some artifacts and strange blobs in the heatmaps. From 0.2 onwards we didn't see such randomish errors anymore. An std of 0.5 is possible in theory and will probably also be fine in terms of the resulting heatmap. Conclusion: at least std of 0.2.

Proposed algorithm:

• use small number of masks for speed (let's say 50)
• repeat:
  • try p_keep value and calculate std
• take p_keep with high enough (highest?) std and use this for a computation with enough masks for quality (2000+?)

Not sure if we should implement this or find some other solution. For instance, let the user mask custom axes in their data for RISE. We should think this through before implementing anything.

An MXNet to ONNX convertor exists as part of the MXNet package, but generates invalid ONNX models in some cases. See also dianna-ai/dianna-exploration#5.

What would be required to support MXNet to ONNX conversion in DIANNA?

Your repository has a workflow which lints your code after every push and when creating a pull request.

Linter workflow may fail if description or keywords field in setup.cfg is empty. Please update these fields. To validate your changes run:

prospector

Enabling githook will automatically lint your code in every commit. You can enable it by running the command below.

git config --local core.hooksPath .githooks

I suggest to stop using this technique. Alternatively, we could stop supporting 3.6 and 3.7. I think we want to support these versions as we hope to build some tool that can be widely used. We might therefore not want to leave out people somehow bound to slightly older python versions.

I see in the method tests that comparisons are done with a pretty high tolerance, e.g. here:

This makes the comparison very lenient, which may cause us to miss changes and/or bugs.

Is it possible to change them to exactly equal tests?

Output of rise can be added repeatedly so you don't have to know the final n_masks in advance.
We can keep adding masks until the output doesn't change much anymore.
Make sure other hyperparameters (p_keep and n_features) have already been optimized (see #24).

Blocked by #56
use
https://github.com/slundberg/shap

When trying out the notebook at example_data/xai_method_study/LRP/LRP_mnist_binary.ipynb, two dependencies are missing: torch and captum. Probably, the same will go for some other notebooks. We should maybe add a requirements.txt at the top level for all dependencies, or add separate ones in each notebook directory with specific deps for those notebooks.

Your Python package should have publicly available documentation, including API documentation for your users.
Read the Docs can host your user documentation for you.

To host the documentation of this repository please perform the following instructions:

1. go to Read the Docs
2. log in with your GitHub account
3. find dianna-ai/dianna in list and press + button.
   • If repository is not listed,
     1. go to Read the Docs GitHub app
     2. make sure dianna-ai has been granted access.
     3. reload repository list on Read the Docs import page
4. wait for the first build to be completed at https://readthedocs.org/projects/dianna/builds
5. check that the link of the documentation badge in the README.md works

See README.dev.md# how to build documentation site locally.

By enabling Zenodo integration, your package will automatically get a DOI which can be used to cite your package. After enabling Zenodo integration for your GitHub repository, Zenodo will create a snapshot and archive each release you make on GitHub. Moreover, Zenodo will create a new DOI for each GitHub release of your code.

To enable Zenodo integration:

1. Go to http://zenodo.org and login with your GitHub account. When you are redirected to GitHub, Authorize application to give permission to Zenodo to access your account.
2. Go to https://zenodo.org/account/settings/github/ and enable Zenodo integration of your repository by clicking on On toggle button.
3. Your package will get a DOI only after you make a release. Create a new release as described in README.dev.md
4. At this point you should have a DOI. To find out the DOI generated by Zenodo:
   1. Visit https://zenodo.org/deposit and click on your repository link
   2. You will find the latest DOI in the right column in Versions box in Cite all versions? section
   3. Copy the text of the link. For example 10.5281/zenodo.1310751
5. Update the badge in your repository

   1. Edit README.md and replace the badge placeholder with the badge link you copied in previous step.
      The badge placeholder is shown below.

      [![DOI](https://zenodo.org/badge/DOI/<replace-with-created-DOI>.svg)](https://doi.org/<replace-with-created-DOI>)

For FAQ about Zenodo please visit https://help.zenodo.org/.

Blocked by #56
use:
https://github.com/slundberg/shap

I would prefer a reorganization of badges on the readme. Thinking about this also made me realize a few things that are missing from the readme.

• The code quality badges should be at the top, just below the title and possibly below a first short descriptive paragraph / tagline. These are: CII checklist, howfairis, Static analysis and coverage. We could also add the CI build badges to this line, but I think these would be better on a next line below the quality badges.
• They should just be buttons, definitely no table, preferably also no descriptive labels.
• There should just be a readme paragraph at the bottom for license/reusing; the license badge can go at the top of that paragraph.
• The community registry badges should be moved to the top of the installation paragraph. Maybe the code repository one can be included there as well; otherwise imho it could be removed.
• We need a How to cite paragraph; we can move the Zenodo DOI badge there as well.
• Docs badge should go at the top of the documentation paragraph.