Giter VIP home page Giter VIP logo

pytest-recording's Issues

Use the plugin on the autouse fixture

Is there a way to use this plugin on the autouse fixture? Scenario: the test introduces a side effect to HTTP service, so it would be good to have autouse setup/teardown which removes this side-effect allowing the whole test suite to run properly when I delete all cassettes and record all of them.

new_episodes doc ?

I wonder what is the intended logic behind the 'new_episodes' option ? I couldn't find it in the README, and I had some "strange" behavior with it...

I ran multiple requests to the same endpoint, in one vcr with --record-mode=new_episodes.
But only the first attempt of each request was registered.

So something like :

import time

import pytest
import aiohttp


@pytest.mark.asyncio
@pytest.mark.vcr()
async def test_multi_call():
    async with aiohttp.ClientSession() as session:
        async with session.get('http://httpbin.org/get') as resp:
            print(resp.status)
            print(await resp.text())

        async with session.get('http://httpbin.org/ip') as resp:
            print(resp.status)
            print(await resp.text())

        time.sleep(1)
        async with session.get('http://httpbin.org/get') as resp:
            print(resp.status)
            print(await resp.text())


        async with session.get('http://httpbin.org/ip') as resp:
            print(resp.status)
            print(await resp.text())

if __name__ == '__main__':
    #pytest.main(['-s', __file__, '--block-network'])
    # record run
    pytest.main(['-s', __file__,'--record-mode=new_episodes'])

Will store only 2 requests/responses, one for http://httpbin.org/get and one for http://httpbin.org/ip.

I was naively expecting to record all requests/responses in one run, and then record new cassettes if a difference between the registered cassettes and the current "run" was found (calling to one more url, or not calling an expected url, etc.)...

So does "episode" means "url" and not "run" ?
Thanks for any info.

How to register matchers?

I would like to register my own matcher.

I looked at pytest-recording's source and noticed vcr is in _vcr.py ; Since it starts with an underscore, I'm guessing that it's not meant to be imported outside of the package?

How do I register my own matcher with pytest-recording?

[BUG]

Describe the bug
Can't write cassette for file uploading. Is it possible to achieve that?

To Reproduce
Steps to reproduce the behavior:

  1. Sample piece of code, which I want to cover with integration test
async def upload_file(filename):
    # some code omitted for simplicity
    data = {'file': open(filename, 'rb')}
    async with aiohttp.ClientSession() as session:
        async with session.post(url, headers=headers, data=data) as response:
            result = await response.read()
            return response.status, result
  1. Test:
@pytest.mark.vcr()
async def test_upload_file():
        img = open(f'{test_data_path}/logo.png', 'rb')
        status, result = await client.upload_file(img)
  1. While test is passed, vcr failed to write cassette:
../venv/lib/python3.8/site-packages/vcr/cassette.py:91: in __exit__
    next(self.__finish, None)
../venv/lib/python3.8/site-packages/vcr/cassette.py:69: in _patch_generator
    cassette._save()
../venv/lib/python3.8/site-packages/vcr/cassette.py:331: in _save
    self._persister.save_cassette(self._path, self._as_dict(), serializer=self._serializer)
../venv/lib/python3.8/site-packages/vcr/persisters/filesystem.py:20: in save_cassette
    data = serialize(cassette_dict, serializer)
../venv/lib/python3.8/site-packages/vcr/serialize.py:58: in serialize
    return serializer.serialize(data)
../venv/lib/python3.8/site-packages/vcr/serializers/yamlserializer.py:15: in serialize
    return yaml.dump(cassette_dict, Dumper=Dumper)
../venv/lib/python3.8/site-packages/yaml/__init__.py:290: in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
../venv/lib/python3.8/site-packages/yaml/__init__.py:278: in dump_all
    dumper.represent(data)
../venv/lib/python3.8/site-packages/yaml/representer.py:27: in represent
    node = self.represent_data(data)
../venv/lib/python3.8/site-packages/yaml/representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
../venv/lib/python3.8/site-packages/yaml/representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
../venv/lib/python3.8/site-packages/yaml/representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
../venv/lib/python3.8/site-packages/yaml/representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
../venv/lib/python3.8/site-packages/yaml/representer.py:199: in represent_list
    return self.represent_sequence('tag:yaml.org,2002:seq', data)
../venv/lib/python3.8/site-packages/yaml/representer.py:92: in represent_sequence
    node_item = self.represent_data(item)
../venv/lib/python3.8/site-packages/yaml/representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
../venv/lib/python3.8/site-packages/yaml/representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
../venv/lib/python3.8/site-packages/yaml/representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
../venv/lib/python3.8/site-packages/yaml/representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
../venv/lib/python3.8/site-packages/yaml/representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
../venv/lib/python3.8/site-packages/yaml/representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
../venv/lib/python3.8/site-packages/yaml/representer.py:48: in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
../venv/lib/python3.8/site-packages/yaml/representer.py:207: in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
../venv/lib/python3.8/site-packages/yaml/representer.py:118: in represent_mapping
    node_value = self.represent_data(item_value)
../venv/lib/python3.8/site-packages/yaml/representer.py:52: in represent_data
    node = self.yaml_multi_representers[data_type](self, data)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <yaml.dumper.Dumper object at 0x10cea75b0>, data = <_io.BufferedReader name='tests/integration/data/logo.png'>

    def represent_object(self, data):
        # We use __reduce__ API to save the data. data.__reduce__ returns
        # a tuple of length 2-5:
        #   (function, args, state, listitems, dictitems)
    
        # For reconstructing, we calls function(*args), then set its state,
        # listitems, and dictitems if they are not None.
    
        # A special case is when function.__name__ == '__newobj__'. In this
        # case we create the object with args[0].__new__(*args).
    
        # Another special case is when __reduce__ returns a string - we don't
        # support it.
    
        # We produce a !!python/object, !!python/object/new or
        # !!python/object/apply node.
    
        cls = type(data)
        if cls in copyreg.dispatch_table:
            reduce = copyreg.dispatch_table[cls](data)
        elif hasattr(data, '__reduce_ex__'):
>           reduce = data.__reduce_ex__(2)
E           TypeError: cannot pickle '_io.BufferedReader' object

../venv/lib/python3.8/site-packages/yaml/representer.py:317: TypeError

Expected behavior
pytest-recording should not fail and provide some mock for IO objects or smth else

Environment (please complete the following information):

  • OS: MacOS
  • Python version: 3.8.2
  • pytest-recording version: 0.10.0
  • pytest version: 6.1.2

Additional context
Add any other context about the problem here.

[BUG] having this plugin enabled breaks Intellij Idea failed test reports

Ok, so this was a weird one to debug...

Simply having the pytest-recording plugin enabled breaks Intellij Idea pytest failed test reports in some specific test cases (here is an example):

  • a test cassette is being recorded via plain vcrpy syntax or via pytest-recording decorator;
  • two (or more) network calls are being executed and recorded: the first one succeeds, the second fails and then an
    error is raised by requests' raise_for_status() method.

Instead of reporting the correct stack trace and main error, Idea reports there has been a failed string comparison
involving url paths.

My guess is that pytest-recording breaks something Idea's test runner relies on to generate errors messages, because:

  • pytest output in the terminal is consistent and correct with or without the plugin installed;
  • disabling the pytest-recording plugin in Idea ui by adding -p no:recording as additional argument restore the correct
    error message;
  • removing the plugin also restore the correct error message.

How to reproduce the issue

Checkout the minimal test repo with git clone https://github.com/CarloDePieri/pytest-recording-idea-issue.

Create a virtualenv and install all dependencies there:

cd pytest-recording-idea-issue
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

IMPORTANT:

  • this DOES NOT install pytest-recording;
  • the test we are going to launch DOES NOT use this plugin decorator, but plain vcrpy.

Then:

  • import the folder into Idea as a new project;
  • add the created virtualenv as a python sdk for the project;
  • run the test from the Idea ui: observe that the test fails with the correct error message reporting a 404;
  • manually install pytest-recording with pip install pytest-recording in the venv;
  • relaunch the test from the Idea ui: the error message is now completely off track: it reports a difference between
    expected and actual values '/api/users/23' != '/api/users/2'.

Under the hood

Idea uses this test runner
to launch pytest tests and generate the report message. Launching the script directly in the terminal shows indeed the
wrong error message when pytest-recording is installed.

Installed software versions

python: 3.10.4
pytest: 7.1.2
pytest-recording: 0.12.0
vcrpy: 4.1.1
requests: 2.27.1
Idea Ultimate: Build #IU-221.5591.52, built on May 10, 2022
os: arch linux

Custom VCR config via separate fixture

Currently, there is no way to specify a module/session-level config without wrapping everything with pytest.mark.vcr.
E.g. test module has 10 tests, 2 of them use pytest.mark.vcr - if they share the same config it will require manual explicit specification on each mark, which means some duplication. An alternative solution is to have a separate fixture of any scope - vcr_config that will be added to the VCR config implicitly (like it is in pytest-vcr).


@pytest.fixture(scope="module")
def vcr_config():
    return {}


@pytest.mark.vcr
def test_a():
    pass


@pytest.mark.vcr
def test_b():
    pass


def test_c():
    pass

In this case, we can avoid VCR setup where it is not needed and the vcr_config fixture could be the first in the merging process - all marks will override colliding keys

Add isort

We need to sort imports in the repo + validate it on CI

Restore handling relative paths in `pytest.mark.vcr`

It was possible in 0.4.0, but now it is not. However, before it was applied only to the first argument in pytest.mark.vcr, other arguments were treated like absolute. It will be nice to support relative paths for all arguments to the vcr mark

[BUG] Default record_mode is 'none' instead of 'once'

Describe the bug
When I use the decorator without passing any record_mode (@pytest.mark.vcr), I get an error saying,

vcr.errors.CannotOverwriteExistingCassetteException: Can't overwrite existing cassette ('/path/test_single.yaml') in your current record mode ('none').

The VCR docs says the default mode is "once" and pytest-recording doesn't mention the default mode, implying it follows VCR's.

To Reproduce
Use the sample test in the docs,

@pytest.mark.vcr
def test_single():
    actualResponse = requests.get("https://httpbin.org/get")
    assert actualResponse.status_code == 200

Expected behaviour
No errors.

Environment (please complete the following information):

  • OS: MacOS
  • Python version: 3.8.7
  • pytest-recording version: 0.11.0
  • pytest version: 6.2.2

[FEATURE] Provide a way to use pytest_recording without `pytest.mark.vcr` decorator

Is your feature request related to a problem? Please describe.
It might be convenient to use _vcr.use_cassette directly for granular control of VCR setup & teardown mechanisms.

Describe the solution you'd like
Create a separate function that will not depend on pytest-related types & expose it as a part of the public API. It might be used by _vcr.use_cassette under the hood (or, likely, _vcr.use_cassette will be renamed to something else, and the new function will take this name).

Additional context
Requested by @selevit

[BUG] record_mode set via vcr_config fixture and marker not honored by block_network

Describe the bug
Setting the record_mode via config fixture nor marker is not honored by block_network. The record_mode variable in block_network is still in default state "none".

To Reproduce

import pytest
import requests


@pytest.fixture(autouse=True)
def vcr_config():
    return {
        "filter_headers": ["authorization"],
        "ignore_localhost": True,
        "record_mode": "once",
    }


@pytest.mark.vcr()
@pytest.mark.block_network(record_mode="once")
def test_request():
    requests.get("https://google.com")

Expected behavior
Shouldn't raise a RuntimeError since we are supposed to be recording.

Environment (please complete the following information):

  • OS: Linux(Manjaro)
  • Python version: 3.8.7
  • pytest-recording version: 0.11.0
  • pytest version: 6.2.4

[FEATURE] Support pytest config file options

Is your feature request related to a problem? Please describe.
Support options in pytest config file

Describe the solution you'd like

[pytest]
block_network = True
ignore_localhost = True
allowed_hosts = [ "::1" ] # Ipv6 localhost isn't considered localhost.

Describe alternatives you've considered
I know I can use addopts, but my addopts line is getting very, very long... ;)

Additional context
Great plugin, love it!

[FEATURE] Record pytest fixtures

Is your feature request related to a problem? Please describe.
In my project i am using fixtures with scope="session" and the requests that are made in the fixture setup are not being recorded, neither in the test casettes, nor in a separate casette, when i mark the fixture with @pytest.mark.vcr

Describe the solution you'd like
I would like it to be possible to decorate fixtures with @pytest.mark.vcr() with the result of a casette being recorded for the individual fixtures, such taht i can use pytest-recording to record fixtures with scope="session"

Describe alternatives you've considered
if i use scope="function" the fixture setup requests are recorded in each test casette and the offline testing works.

[FEATURE] Parameterize vcr_config

Is your feature request related to a problem? Please describe.

I'm working on a library that supports multiple web services behind a single interface.

Now I've written some tests for that interface and would like to run them for each of the web services I support. This is where pytest's fixture parameterization shines and it works well without VCR in the picture, the implementations simply make real requests.

Obviously that's not ideal, so I wanna use VCR. I need different configurations for each web service though. Unfortunately I can't find a way to parameterize vcr_config.

Describe the solution you'd like

No idea, sorry.

Describe alternatives you've considered

I guess I could use the vcr fixture somehow?

What is the relationship to pytest-vcr?

I have seen pytest-recording (on pypi) which looks pretty similar:

  • Uses @pytest.mark.vcr
  • Both seem to be wrappers around vcr.py
  • 49 stars, 3 forks, 2 contributors vs 67 stars, 17 forks, 7 contributors
  • First tag in 2019 vs first tag in 2017

My guess is that pytest-recording is a fork of pytest-vcr. But why was the fork done and what is the relationship to pytest-vcr? When should one use one or the other?

[FEATURE] Delete casette if test fails when running with --record-mode=once

Is your feature request related to a problem? Please describe.
When I'm building a new test, I'll run the test with --record-mode=once, but the test might fail (since it's new) and I then have to go delete the file so it'll re-record. It's kinda an annoyance to go find and make sure I delete the right file.

Describe the solution you'd like
I'd love it if there was some logic where when running with --record-mode=once it only saved the files from the tests that pass, then I could write a bunch of new tests, run them all, the good ones would get saved, and then I'd go fix the ones that didn't pass.

Just kinda a nice to have I'd think. Definitely not a big issue. Though it seems odd to save a cassette for a failed test in general.

Describe alternatives you've considered
Considered --record-mode=any, but that would re-run unrelated tests that I'm not working on.

Improve Readme

Readme doesn't document requirements, setup process, etc.

I've installed pytest-recording and hacking around now to get it working but would be great if this was all part of the Readme.

Write multiple tests to the same cassette

Is it possible to write multiple tests to the same cassette? If multiple tests are making the same API calls, it's inefficient to record them again and again, better to use a shared cassette.

This is possible with vcr.use_cassette, but doesn't seem to be possible with pytest.mark.vcr, since the default cassette name is always the test name.

For example:

@pytest.mark.vcr("tests/cassettes/shared_cassette.yaml")
def test_a(): ...

@pytest.mark.vcr("tests/cassettes/shared_cassette.yaml")
def test_b(): ...

In this example, you will still write new cassettes to tests/cassettes/test_a.yaml and tests/cassettes/test_b.yaml, rather than using the shared one.

Simplify working with network blocker

Probably two functions will be helpful - enable / disable, in some cases context manager is not needed and calling __enter__ / __exit__ manually is not convenient

Doesn't seem to work with requests inside a Twisted Reactor process

Hi, I'm new to Python so apologies in advance if this is obvious or I don't explain things very well.

For a learning project I'm writing a web scraper with tests. In the following example it works as expected with the commented out line but requests inside the CrawlerProcess/Twisted Reactor aren't being picked up.

    @pytest.mark.vcr
    def test_parse(self):
        # assert requests.get("http://httpbin.org/ip").text == '{"ip": true}'
        BaseSpider.start_urls = ['thoughtbot']

        BaseSpider.custom_settings = {
            'ROBOTSTEXT_OBEY': False
        }

        process = CrawlerProcess()
        process.crawl(BaseSpider)
        process.start()

I'm not sure if this is a bug or I'm just doing something wrong?

[FEATURE] Run Python 3.9 builds

  • Add 39 to envlist in tox.ini
  • Add 39 to depends of the coverage-report job in tox.ini
  • Add Programming Language :: Python :: 3.9 to the classifiers list in setup.py
  • Add Python 3.9 job in .github/workflows/build.yml similarly to other builds

Consider always keeping the cassette with default name

For example if there are a test and a cassette for it and then we need to add some common data:

@pytest.mark.vcr("extra.yaml")
def test_something():
    pass

In this case, we will look only into extra.yaml because of this - https://github.com/kiwicom/pytest-recording/blob/master/src/pytest_recording/plugin.py#L84

However, it is useful to be able to add some extra common data and still using some specific cassette for the test (named by the default rules). Now it is possible only with specifying it manually in the vcr mark

Use default mode `once` in README

I'm new to VCR.py and pytest-recording. Just setup for my scraper project today.

Followed by the README doc, I used --record-mode=all to run pytest. However, the tests will failed in CI (I used CircleCI) and the YAMLs under cassettes will be changed.

After read VCR.py doc, I change to --record-mode=once. Now all the cassettes won't be updated and all tests passed in CI.

IMO, I suggest to use --record-mode=once in README, which may be easier for beginner.

Stripe library (urlfetch) just waits with `block_network`

Hey,

Thanks a lot for this library!

I managed to get everything working, except that when I run our test suite with block_network it just waits and nothing happens.

VCR functionality does work even if I don't pass the flag so I can run the tests with recorded network communication data, but it seems for some reason the Stripe library (using urlfetch apparently) just waits when the network is blocked.

I guess maybe #16 would solve this issue?

How to setup common VCR config for all tests

The following config seems not working under tests/__init__.py, but works within tests/test_foo.py, tests/test_bar.py...

import pytest


@pytest.fixture(scope='module', autouse=True)
def vcr_config():
  return {'ignore_hosts': ['oauth2.googleapis.com']}

[Bug] Using `once` record_mode gives error

Problem

Using once gives an error:

vcr.errors.CannotOverwriteExistingCassetteException: Can't overwrite existing cassette ([PATH_OF_CASETTE_FILE]) in your current record mode ('once').
E               No match for the request (<Request (GET) https://postman-echo.com/get?foo1=bar1&foo2=bar2>) was found.
E               No similar requests, that have not been played, found.

I ran this the first time so PATH_OF_CASETTE_FILE shouldn't exist although the error seems to say it exists. I checked the PATH_OF_CASETTE_FILE that the error specifies before I run my test the 2nd time and indeed no such file exists.

To Reproduce

import pytest
import requests

@pytest.mark.vcr(record_mode="once")
def test_vcr():
    response = requests.get("https://postman-echo.com/get?foo1=bar1&foo2=bar2")
    assert response.status_code == 200

Stacktrace

Click to expand!
../../Library/Caches/pypoetry/virtualenvs/featuredproductsswitch-q2O7GQgj-py3.8/lib/python3.8/site-packages/requests/api.py:76: in get
    return request('get', url, params=params, **kwargs)
../../Library/Caches/pypoetry/virtualenvs/featuredproductsswitch-q2O7GQgj-py3.8/lib/python3.8/site-packages/requests/api.py:61: in request
    return session.request(method=method, url=url, **kwargs)
../../Library/Caches/pypoetry/virtualenvs/featuredproductsswitch-q2O7GQgj-py3.8/lib/python3.8/site-packages/requests/sessions.py:530: in request
    resp = self.send(prep, **send_kwargs)
../../Library/Caches/pypoetry/virtualenvs/featuredproductsswitch-q2O7GQgj-py3.8/lib/python3.8/site-packages/requests/sessions.py:643: in send
    r = adapter.send(request, **kwargs)
../../Library/Caches/pypoetry/virtualenvs/featuredproductsswitch-q2O7GQgj-py3.8/lib/python3.8/site-packages/requests/adapters.py:439: in send
    resp = conn.urlopen(
../../Library/Caches/pypoetry/virtualenvs/featuredproductsswitch-q2O7GQgj-py3.8/lib/python3.8/site-packages/urllib3/connectionpool.py:670: in urlopen
    httplib_response = self._make_request(
../../Library/Caches/pypoetry/virtualenvs/featuredproductsswitch-q2O7GQgj-py3.8/lib/python3.8/site-packages/urllib3/connectionpool.py:417: in _make_request
    httplib_response = conn.getresponse(buffering=True)

TypeError when using network blocking with address as bytes

pattern = '(127.0.0.2)', string = b'127.0.0.1', flags = 0

    def match(pattern, string, flags=0):
        """Try to apply the pattern at the start of the string, returning
        a Match object, or None if no match was found."""
>       return _compile(pattern, flags).match(string)
E       TypeError: cannot use a string pattern on a bytes-like object

bytearray is also a possible type

And the code to trigger:

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
    sock.connect((b"127.0.0.1", 80))

Disallow multiple `network.block()` calls

If the --block-network CMD option is passed, and if network.block is called in code, then one will overwrite another, and it depends on the execution order, which may lead to hard-to-track bugs in tests (it happened in one of our projects)

Implementation notes:

  • In the network.block call check if socket.socket.connect is already replaced
  • If so, raise a RuntimeError with some explanation

Control what components to patch in `block_network`

Sometimes it is needed to use only one or another library to access the network. E.g., sometimes socket is ok, but pycurl is not, or in the case if there will be more libraries to patch (maybe psycopg2 or some others), then it would be nice to have some granular control over what to disable and what is not.

Implementation notes:

  • Add a global container of available transports, e.g. TRANSPORTS = ("socket", "pycurl")
  • Then add a new transports argument to network.block with the default value of TRANSPORTS
  • Add conditions inside network.block to patch only transports defined in the new arguments

Cassettes ordering can be changed in the multi-cassette mode

You use startmap to make the cassette list unique, but it leads to the situation when ordering of the cassettes can change, because map keys isn't ordered. I faced with this problem, when my tests failed in CI, but worked locally because of this.

requests, responses = starmap(unpack, zip(*all_content))

BTW, what for do you make the cassette list unique? I would like to put one cassette multiple times in one testcase, so ensure that the same request is performed many times. Maybe just get rid of this?

P.S. Thanks a lot for the great lib, I really like it!

Decode compressed response

I have the following in my conftest.py:

@pytest.fixture(scope="module")
def vcr_config():
    return {
        "cassette_library_dir": f"{TESTS_DIR}/fixtures/cassettes",
        "decode_compressed_response": True,
        "filter_headers": ["authorization"]
    }

But decoding is happening intermittently.

Flag to turn on/off vcr

Hi,
I'd like to have some flag (the best option env variable), to turn on/off vcr for tests marked with @pytest.mark.vcr.
By default it should work as is. But if I set env variable "USE_VCR=false" it should run real http requests and no cassetes should be rewritten. Or it is possible to achieve somehow with CLI options, custom decorators, etc?

More robust message for network access blocking

Hi!
It would be great to see a more robust error message than "Network is disabled".
This would be especially useful for newcomers or people who aren't aware that this package is installed on the project they are working on.

Some ideas for what it could contain:

  1. A title saying that the error is coming from this package
  2. A short summary of what happened. Why the error was raised.
  3. The http request that was made
  4. The list of configured allowed hosts (for quick comparison with the actual request made)
  5. Possible ways to solve the issue (change the request, record a cassette, add allowed hosts...)
  6. A link to the relevant section in the docs.

A great example of a similar error message is the message in Ruby's VCR implementation which you can see here: https://stackoverflow.com/questions/25799165/vcr-throwing-cant-handle-request-on-the-second-attempt-to-run-testfirst-attempt

Thanks!

[FEATURE] Tap into additional vcr_config parameters in block_network for allowed_hosts

This is kinda a chimeric issue. Related to my #68, block_network should also have the option to set allowed_hosts via vcr_config, imho.

But then I thought "aren't the ignore_localhost and ignore_hosts parameters from the vcr_config basically the same as allowed_hosts?". VCRpy is adding localhost, 0.0.0.0 and 127.0.0.1 if ignore_localhostis truthy, and also adds ignore_hosts to the set.

I think it would make for a more seamless integration with vcrpy if allowed_hosts is also using ignore_localhost and ignore_hosts from the vcr_config to construct the list while retaining backwards compatibility :-)

Host names are not working in `allowed_hosts`

Example from the README file:

import pytest
import requests

@pytest.mark.block_network(allowed_hosts=["httpbin.*"])
def test_access():
    assert requests.get("http://httpbin.org/get").text == '{"get": true}'
    with pytest.raises(RuntimeError, match=r"^Network is disabled$"):
        requests.get("http://example.com")

It will fail on the first line:

self = <socket.socket fd=10, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('0.0.0.0', 0)>, address = ('100.25.11.135', 80), args = (), kwargs = {}, host = '100.25.11.135'

    def network_guard(self, address, *args, **kwargs):
        host = ""
        if self.family in (socket.AF_INET, socket.AF_INET6):
            host = address[0]
        elif self.family == socket.AF_UNIX:
            host = address
        if is_host_in_allowed_hosts(host, allowed_hosts):
            return original_func(self, address, *args, **kwargs)
>       raise RuntimeError("Network is disabled")
E       RuntimeError: Network is disabled

[FEATURE] A way to allow all local IPs in network blocking context

Is your feature request related to a problem? Please describe.
When running in docker-compose you need to resolve local services first (e.g. DB) and then pass them to --allowed-hosts.

Describe the solution you'd like
Add --allow-local-ips flag to CLI + related arg to the block mark to allow requests to local IPs.
Use something like ipaddress.is_local to check all IPs in such a case.

Additional context
The feature & approach was proposed by @selevit

[FEATURE] usage for doctests

Foreword: thank you for this plugin, I like the API, and it is really useful.

Is your feature request related to a problem? Please describe.
The plugin works very well for normal tests, but I'd really like to use it with doctests.
I run those with pytest --doctest-modules and/or pytest --doctest-glob="*.rst". I can define a conftest.py file, but in it I can define fixtures, and not tests, so I cannot use the pytest.mark.vcr

Describe the solution you'd like
I'd like to be able to configure the tests to use a cassette/several cassettes for the tests in (say) the README and the modules, and then the pytest --doctest-modules would record and use the cassette instead of calling the network.

Describe alternatives you've considered
I can add this in conftest.py:

from vcr import use_cassette

@pytest.fixture(autouse=True)
def with_cassette():
    with use_cassette("tests/doctests/cassette.yaml", record_mode="once"):
        yield

But this does not respect whatever will be passed to the command line arguments as it hardcodes the cassette and record mode. If it creates a cassette for each file, that would be better, instead of using one big cassette.

Don't add YAML extension to JSON cassettes

Thank you for the plugin. Unfortunately, it is really hard to find it as pytest-vcr always comes first in the search.

I have the following issue:
If I use serializer="json" and I use something like @pytest.mark.vcr('list_rooms.json'), then the plugin adds yaml extension to the cassette (list_rooms.json.yaml), though it is saved as json correctly.

Fuzzy cassettes

For unhappy path testing, there could be an extra parameter that will mutate recorded cassettes.
Use cases:

  • Check how the app will work if the format of responses will change
  • Validate error handling

Possible fuzzes:

  • Mutate response body. JSON - add/remove fields in objects / lists. Change values. Make JSON invalid.
  • Completely change response content type
  • Change response status code
  • Mutate requests during recording??
  • Raise exceptions instead of real responses
  • Add delays to responses

All these could be combined and generate test cases

Provide extra config option for shared cassettes dir(s)

It could be a separate fixture that will return a list of paths where to look for cassettes. Currently, we need to use the full path to the cassette in pytest.mark.vcr which is tedious. However, shared cassettes could be used via a separate mark to avoid breaking the existing interface - pytest.mark.vcr_shared("first.yaml", "second.yaml")

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.