Test for differentiability about torchmetrics HOT 10 CLOSED

@bhadreshpsavani please feel free to open a PR with what you got, then we can better help out :]

from torchmetrics.

Hi @SkafteNicki and @Borda,
Can I take this?

from torchmetrics.

sure, open a draft PR and we can help you there 🐰
to run tests locally, you can use make test which is running following command under the hood
https://github.com/PyTorchLightning/metrics/blob/fb0ee3ff0509fdb13bd07b6aac3e20c642bb5683/Makefile#L20

from torchmetrics.

Hi! thanks for your contribution!, great first issue!

from torchmetrics.

@bhadreshpsavani please feel free to do so :]
Ping me either here or on our slack channel if you need any help.

from torchmetrics.

@bhadreshpsavani for the metrics that ware not differentiable, please also wrap the forward and compute functions with torch.no_grad to reduce the memory footprint.

from torchmetrics.

@bhadreshpsavani how is it going, have you started on adding these tests?

from torchmetrics.

Hi @Borda,
I have made changes for two metrics. I was not very confident about the changes. Since I don't know how to run that test cases and check the changes. I am not very sure I did it the right way.

Shall I create a PR so someone can review it and suggest?

from torchmetrics.

I have made a pull request but a lot of tests are failing,
I try to test the changes using the below commands make test, python -m pytest torchmetrics tests -v --cov=torchmetrics before making changes but it was giving an error.

(metrics-env) D:\metrics>make test
FIND: Parameter format not correct
# clean all temp runs
process_begin: CreateProcess(NULL, # clean all temp runs, ...) failed.
make (e=2): The system cannot find the file specified.
make: *** [Makefile:7: clean] Error 2

and

(metrics-env) D:\metrics>python -m pytest torchmetrics tests -v --cov=torchmetrics
ERROR: usage: __main__.py [options] [file_or_dir] [file_or_dir] [...]
__main__.py: error: unrecognized arguments: --cov=torchmetrics
  inifile: D:\metrics\setup.cfg
  rootdir: D:\metrics

from torchmetrics.

Hi @SkafteNicki,
Is it possible that for few metrics can't have this property?
I was trying to implement this for cohen_kappa and ``avarage_precision` i wrote code but few test cases I wrote were failing. I could not figure out how to fix it.
Let me create PR for that so that you can review the changes.

from torchmetrics.