[torchlib] Fix special.erfcx about onnxscript HOT 5 OPEN

Summary

The output of ONNX Runtime does not match that of PyTorch when executing test
ops_test.TestOutputConsistencyFullGraphCPU.test_output_match_opinfo__special_erfcx_cpu_float32, sample 0 in ONNX Script TorchLib.

To recreate this report, use

CREATE_REPRODUCTION_REPORT=1 python -m pytest onnxscript/tests/function_libs/torch_lib/ops_test.py -k test_output_match_opinfo__special_erfcx_cpu_float32

Inputs

Shapes: ['Tensor<torch.Size([20]), dtype=torch.float32>']

kwargs = {}
inputs = (tensor([ 6.8808,  7.4701, -2.1085,  8.2675, -1.9719,  1.8161, -4.3817,  5.2855,
         7.9339, -6.6027,  7.8228,  1.6844,  6.6493,  1.2189,  4.3397, -1.2707,
         6.9380,  1.3303, -4.2016,  2.2941]),)

Expected output

Shape: torch.Size([20])

expected = tensor([8.1154e-02, 7.4867e-02, 1.7025e+02, 6.7753e-02, 9.7418e+01, 2.7655e-01,
        4.3567e+08, 1.0493e-01, 7.0559e-02, 1.7144e+19, 7.1546e-02, 2.9381e-01,
        8.3921e-02, 3.7443e-01, 1.2680e-01, 9.6895e+00, 8.0499e-02, 3.5172e-01,
        9.2827e+07, 2.2725e-01])

Actual output

Shape: torch.Size([20])

actual = tensor([0.0000e+00, 0.0000e+00, 1.7025e+02, 0.0000e+00, 9.7418e+01, 2.7655e-01,
        4.3567e+08, 0.0000e+00, 0.0000e+00, 1.7144e+19, 0.0000e+00, 2.9381e-01,
        0.0000e+00, 3.7443e-01, 0.0000e+00, 9.6895e+00, 0.0000e+00, 3.5172e-01,
        9.2827e+07, 2.2724e-01])

Difference

--- actual
+++ expected
@@ -1,4 +1,4 @@
-tensor([0.0000e+00, 0.0000e+00, 1.7025e+02, 0.0000e+00, 9.7418e+01, 2.7655e-01,
-        4.3567e+08, 0.0000e+00, 0.0000e+00, 1.7144e+19, 0.0000e+00, 2.9381e-01,
-        0.0000e+00, 3.7443e-01, 0.0000e+00, 9.6895e+00, 0.0000e+00, 3.5172e-01,
-        9.2827e+07, 2.2724e-01])
+tensor([8.1154e-02, 7.4867e-02, 1.7025e+02, 6.7753e-02, 9.7418e+01, 2.7655e-01,
+        4.3567e+08, 1.0493e-01, 7.0559e-02, 1.7144e+19, 7.1546e-02, 2.9381e-01,
+        8.3921e-02, 3.7443e-01, 1.2680e-01, 9.6895e+00, 8.0499e-02, 3.5172e-01,
+        9.2827e+07, 2.2725e-01])

Full error stack

Tensor-likes are not close!

Mismatched elements: 9 / 20 (45.0%)
Greatest absolute difference: 0.12679913640022278 at index (14,) (up to 1e-05 allowed)
Greatest relative difference: 1.0 at index (0,) (up to 1.3e-06 allowed)
  File "/home/justinchu/dev/onnx-script/onnxscript/tests/function_libs/torch_lib/ops_test.py", line 266, in run_test_output_match
    torch.testing.assert_close(
  File "/home/justinchu/anaconda3/envs/onnx/lib/python3.10/site-packages/torch/testing/_comparison.py", line 1520, in assert_close
    raise error_metas[0].to_error(msg)

from onnxscript.

Summary

The output of ONNX Runtime does not match that of PyTorch when executing test ops_test.TestOutputConsistencyFullGraphCPU.test_output_match_opinfo__special_erfcx_cpu_float32, sample 0 in ONNX Script TorchLib.

To recreate this report, use

CREATE_REPRODUCTION_REPORT=1 python -m pytest onnxscript/tests/function_libs/torch_lib/ops_test.py -k test_output_match_opinfo__special_erfcx_cpu_float32

Inputs

Shapes: ['Tensor<torch.Size([20]), dtype=torch.float32>']

Details

Expected output

Shape: torch.Size([20])

Details

Actual output

Shape: torch.Size([20])

Details

Difference

Details

Full error stack

Tensor-likes are not close!

Mismatched elements: 9 / 20 (45.0%)
Greatest absolute difference: 0.12679913640022278 at index (14,) (up to 1e-05 allowed)
Greatest relative difference: 1.0 at index (0,) (up to 1.3e-06 allowed)
  File "/home/justinchu/dev/onnx-script/onnxscript/tests/function_libs/torch_lib/ops_test.py", line 266, in run_test_output_match
    torch.testing.assert_close(
  File "/home/justinchu/anaconda3/envs/onnx/lib/python3.10/site-packages/torch/testing/_comparison.py", line 1520, in assert_close
    raise error_metas[0].to_error(msg)

Should we raise an issue there?

from onnxscript.

This is our own issue in torchlib I believe. The implementation is not numerically stable.

from onnxscript.

I feel like it's either PyTorch impl (math) or ORT ops precision issue.

from onnxscript.

Next step: https://github.com/pytorch/pytorch/blob/bf20b56e9d40dd005399fce05b3a073d8132e04f/aten/src/ATen/native/Math.h#L2152

from onnxscript.