Pytest keeps reference to attributes of failed test cases for too long about pytest HOT 4 OPEN

this is a unfortunate side-effect of ensuring staying debugable

there was no effort yet to provide a api/hook to purge tracebakcs/stacktraces of expensive resources

in this case im considering the array a "resource" as its something that puts possibly large pressure onto the system and ought to be purged if disposable

from pytest.

Hi @RonnyPfannschmidt,
Thanks a lot for the follow-up.
Wondering what was the reason for not providing an api/hook to remove the traceback?
Or the pytest team does not have this in plans at all?
Is it complicated for a beginner to pytest repo like me to try to implement it?

from pytest.

For others who also encounter this issue in their tests,
I found a work-around

import weakref
import numpy as np
import pytest

N = 20_000_000

def get_data(n):
    d = np.random.rand(n,100)
    return d

def process_data(d):
    return np.diff(d)

def test1(data_provider):
    d = data_provider.get(N)
    process_data(d)
    assert False
    
def test2():
    d = get_data(N)
    process_data(d)
    
class DataFlyweight:
    def __init__(self):
        self._data =[]
        
    def get(self,n):
        if n not in self._data:
            self._data.append(get_data(n))
        return weakref.proxy(self._data[-1])

    def clear(self):
        self._data.clear()
    
@pytest.fixture
def data_provider():
    data_flyweight = DataFlyweight()
    yield data_flyweight
    data_flyweight.clear()

Memory does not explode and the second test passed as expected.

from pytest.

Multiple reasons

Initially tracebacks where not attached to exceptions and extracting ex info typically lost a number of locals automatically

Also for most test suites memory intensive variables where no issue

So there was no incentive/pressure to work on a feature with limited win and excessive possible edge cases for a long time

from pytest.