instagram / monkeytype Goto Github PK

E.g. someone should be able to implement support for django.utils.functional.cached_property via config, we shouldn't need to hardcode support in core.

By default MonkeyType respects existing annotations as the authority, and uses them over annotations inferred from traces when generating stubs.

For checking correctness of existing annotations against recorded traces, it would be useful to have an option to generate a stub that only considers the traces and ignores existing annotations.

Trying to set up dev environment per instructions in CONTRIBUTING.rst

Getting an import error:
Traceback (most recent call last):
File "/home/johnar/github/MonkeyType/monkeytype/cli.py", line 17, in
from typing import (
File "/home/johnar/github/MonkeyType/monkeytype/typing.py", line 10, in
from typing import (
ImportError: cannot import name 'Any'

In pycharm it actually shows that import _Any and _Union result in unresolved reference, as they're not defined in the typing module.

This is with stdlib typing 3.6.4:
typing 3.6.4

After running monkeytype.exe run on my script (Windows 10, python 3.6.2), I get the error below:

Notice that the .sql file is dumped, and the error only occurs after my script is executed.

ERROR:monkeytype.tracing:Failed collecting trace
Traceback (most recent call last):
File "c:\program files\python36\lib\site-packages\monkeytype\tracing.py", li ne 252, in call
self.handle_return(frame, arg)
File "c:\program files\python36\lib\site-packages\monkeytype\tracing.py", li ne 227, in handle_return
typ = get_type(arg)
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 76, in get_type
val_type = shrink_types(get_type(v) for v in obj.values())
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 34, in shrink_types
types = tuple(types)
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 76, in
val_type = shrink_types(get_type(v) for v in obj.values())
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 62, in get_type
return Type[obj]
File "c:\program files\python36\lib\typing.py", line 652, in inner
return func(*args, **kwds)
File "c:\program files\python36\lib\typing.py", line 1098, in getitem
params = tuple(_type_check(p, msg) for p in params)
File "c:\program files\python36\lib\typing.py", line 1098, in
params = tuple(_type_check(p, msg) for p in params)
File "c:\program files\python36\lib\typing.py", line 376, in _type_check
raise TypeError("Plain %s is not valid as type argument" % arg)
TypeError: Plain typing.Generic is not valid as type argument
ERROR:monkeytype.tracing:Failed collecting trace
Traceback (most recent call last):
File "c:\program files\python36\lib\site-packages\monkeytype\tracing.py", li ne 252, in call
self.handle_return(frame, arg)
File "c:\program files\python36\lib\site-packages\monkeytype\tracing.py", li ne 227, in handle_return
typ = get_type(arg)
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 76, in get_type
val_type = shrink_types(get_type(v) for v in obj.values())
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 34, in shrink_types
types = tuple(types)
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 76, in
val_type = shrink_types(get_type(v) for v in obj.values())
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 62, in get_type
return Type[obj]
File "c:\program files\python36\lib\typing.py", line 652, in inner
return func(*args, **kwds)
File "c:\program files\python36\lib\typing.py", line 1098, in getitem
params = tuple(_type_check(p, msg) for p in params)
File "c:\program files\python36\lib\typing.py", line 1098, in
params = tuple(_type_check(p, msg) for p in params)
File "c:\program files\python36\lib\typing.py", line 376, in _type_check
raise TypeError("Plain %s is not valid as type argument" % arg)
TypeError: Plain typing.Generic is not valid as type argument
ERROR:monkeytype.tracing:Failed collecting trace
Traceback (most recent call last):
File "c:\program files\python36\lib\site-packages\monkeytype\tracing.py", li ne 252, in call
self.handle_return(frame, arg)
File "c:\program files\python36\lib\site-packages\monkeytype\tracing.py", li ne 227, in handle_return
typ = get_type(arg)
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 76, in get_type
val_type = shrink_types(get_type(v) for v in obj.values())
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 34, in shrink_types
types = tuple(types)
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 76, in
val_type = shrink_types(get_type(v) for v in obj.values())
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 62, in get_type
return Type[obj]
File "c:\program files\python36\lib\typing.py", line 652, in inner
return func(*args, **kwds)
File "c:\program files\python36\lib\typing.py", line 1098, in getitem
params = tuple(_type_check(p, msg) for p in params)
File "c:\program files\python36\lib\typing.py", line 1098, in
params = tuple(_type_check(p, msg) for p in params)
File "c:\program files\python36\lib\typing.py", line 376, in _type_check
raise TypeError("Plain %s is not valid as type argument" % arg)
TypeError: Plain typing.Generic is not valid as type argument

We currently can't rehydrate builtins.traceback.

I've run monkeytype run manage.py test in my django project after pip install monkeytype (inside of a virtual env) and I'm not seeing any output of the sqllite file described in the README.

Running monkeytype stub for any of my modules doesn't work either. If there's something different that I have to do for django tests, let's add that to the README. Otherwise, am I missing something?

Currently we generate e.g. WARNING: Failed decoding trace for function ...: Module 'django.utils.functional' has no attribute 'lazy.<locals>', but it should be possible to special-case this and extract the contained type.

Report the number of samples that were used to generate a method's stub. This will help gauge confidence in the results.

I want to add types to myscript.py,

so I do monkeytype run myscript.py

but monkeytype stub myscript.py says No traces found

so I dumped monkeytype.sqlite3 and see it has traces under the module named __main__,

but monkeytype stub __main__ says ERROR: Failed decoding trace: Module '__main__' has no attribute 'main'

See #48 for a reproducible example case. Running retype directly shows a useful error, but somehow this doesn't get passed through to MonkeyType output, even though we pipe both stdout and stderr in the subprocess call. Need to investigate why not.

The stubs generated by monkeytype don't preserve annotations that use type aliases. For example, if a function has the following signature:

Number = Union[int, float]

def sum(a: Number, b: Number) -> Number:
    return a + b

MonkeyType will expand Number. The generated stub will be:

def sum(a: Union[int, float], b: Union[int, float]) -> Union[int, float]: ...

Attempting to apply a stub like this will fail with an error from retype of the form incompatible annotation for ....

I'm not sure if there is anything we can do about this. One work around would be an option to omit functions that already have annotation from the generated stubs.

Thx for this awesome project!
This was the code which threw the error:

def get_types():

    class Type(object):
        def __init__(self, name, slug):
            self.name = name
            self.slug = slug

    return [Type(type_,type_.lower())
            for type_ in get_types_from_somewhere_else()]

$ monkeytype run
Traceback (most recent call last):
File "/usr/local/bin/monkeytype", line 7, in
from monkeytype.cli import entry_point_main
File "/usr/local/lib/python3.5/dist-packages/monkeytype/init.py", line 7, in
from typing import (
ImportError: cannot import name 'ContextManager'

Please follow your own practices espoused in https://code.facebook.com/posts/300798627056246/relicensing-react-jest-flow-and-immutable-js/

Re-license me maybe?

The newer version of MonkeyType does not work in IPython. I think e8bb8a6 could be the reason.

An interactive session in IPython will assert __name__ == '__main__' it would still be meaningful to capture the CallTrace so folks could interactively type their notebook/python source.

I can totally understand if this falls outside the scope of the project.

Everything I try to do with the monkeytype command-line took gives me this:

Traceback (most recent call last):
  File "/Users/guido/v36/bin/monkeytype", line 7, in <module>
    from monkeytype.cli import entry_point_main
  File "/Users/guido/v36/lib/python3.6/site-packages/monkeytype/__init__.py", line 12, in <module>
    from monkeytype.config import (
  File "/Users/guido/v36/lib/python3.6/site-packages/monkeytype/config.py", line 83, in <module>
    venv_real_prefix = getattr(sys, 'real_prefix')
AttributeError: module 'sys' has no attribute 'real_prefix'

This is with a venv running Python 3.6.2 on Mac. Maybe you meant getattr(sys, 'real_prefix', None) ?

https://www.python.org/dev/peps/pep-0498/

def a(b):
    return f'b == {b}'

Currently, monkeytype errors on apply with

error: .../monkey/test.py: Cannot parse: 2:12:     return f'b == {b}'

I've seen this #25 and tried to invoke pytest command as monkeytype run pytest but this results in following error

Traceback (most recent call last):
  File "/home/user/.local/share/virtualenvs/venv/bin/monkeytype", line 11, in <module>
    sys.exit(entry_point_main())
  File "/home/user/.local/share/virtualenvs/venv/lib/python3.6/site-packages/monkeytype/cli.py", line 257, in entry_point_main
    sys.exit(main(sys.argv[1:], sys.stdout, sys.stderr))
  File "/home/user/.local/share/virtualenvs/venv/lib/python3.6/site-packages/monkeytype/cli.py", line 245, in main
    handler(args, stdout, stderr)
  File "/home/user/.local/share/virtualenvs/venv/lib/python3.6/site-packages/monkeytype/cli.py", line 130, in run_handler
    runpy.run_path(args.script_path, run_name='__main__')
  File "/usr/lib64/python3.6/runpy.py", line 261, in run_path
    code, fname = _get_code_from_file(run_name, path_name)
  File "/usr/lib64/python3.6/runpy.py", line 231, in _get_code_from_file
    with open(fname, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'pytest'

Right now it just annotates those as OrderedDict, instead of specifying the keys and values types like it does with regular dicts

I want to run

myscript update

but monkeytype run myscript update

errors with:

monkeytype: error: unrecognized arguments: update

e.g. should not generate from _io import BytesIO in a stub

Given

UserId = NewType('UserId', int)

def get_user(self, user_id: UserId) -> 'User':

MonkeyType generates

def get_user(
    self,
    user_id: <function NewType.<locals>.new_type at 0x1090a52f0>
) -> 'User': ...

which then causes retype to fail with this error message:

error: /path/to/module.py: invalid syntax (<unknown>, line 18)

This happens even when --exclude-unparsable-defaults is passed.

I was able to come up with a solution to this, which I'll submit a PR for. However, it seems hacky to me and I'd appreciate feedback on it.

Storing only unique traces in the logger has a couple advantages:

1. Reduces memory overhead of tracing.
2. Should improve annotation quality by improving diversity of queried traces.

There doesn't seem to be much use case for seeing the actual distribution of traces, but if someone needs that, it's only a few lines of code to implement their own logger.

The end of the traceback looks like:

  File "/.../lib/python3.6/site-packages/monkeytype/typing.py", line 180, in rewrite_Union 
    return Union[elems] 
  File "/.../lib/python3.6/typing.py", line 682, in inner 
    return func(*args, **kwds) 
  File "/.../lib/python3.6/typing.py", line 793, in __getitem__ 
    raise TypeError("Cannot take a Union of no types.") 
TypeError: Cannot take a Union of no types. 

Haven't worked on a repro, but from looking at the code I think it ought to be pretty easy to repro with e.g. a List parameter that is only ever called with an empty list. In this case we should just leave the annotation as List[Any], there's nothing else we can do.

git clone https://github.com/celery/celery
cd celery/
python setup.py test # for quick install of test dependencies
monkeytype run setup.py test

Failed collecting trace
Traceback (most recent call last):
  File "/private/tmp/tmp2/celery/venv/lib/python3.6/site-packages/monkeytype/tracing.py", line 252, in __call__
    self.handle_return(frame, arg)
  File "/private/tmp/tmp2/celery/venv/lib/python3.6/site-packages/monkeytype/tracing.py", line 227, in handle_return
    typ = get_type(arg)
  File "/private/tmp/tmp2/celery/venv/lib/python3.6/site-packages/monkeytype/typing.py", line 76, in get_type
    val_type = shrink_types(get_type(v) for v in obj.values())
  File "/private/tmp/tmp2/celery/venv/lib/python3.6/site-packages/monkeytype/typing.py", line 39, in shrink_types
    return Union[types]
  File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/typing.py", line 679, in inner
    return cached(*args, **kwds)
  File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/typing.py", line 803, in __getitem__
    return self.__class__(parameters, origin=self, _root=True)
  File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/typing.py", line 743, in __new__
    parameters = _remove_dups_flatten(parameters)
  File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/typing.py", line 647, in _remove_dups_flatten
    for t2 in all_params - {t1}
  File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/typing.py", line 649, in <genexpr>
    t2.__origin__ is not None)):
  File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/typing.py", line 1153, in __subclasscheck__
    return super().__subclasscheck__(cls)
  File "/usr/local/bin/../Cellar/python3/3.6.3/bin/../Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py", line 207, in __subclasscheck__
    ok = cls.__subclasshook__(subclass)
  File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/typing.py", line 884, in __extrahook__
    if issubclass(subclass, scls):
  File "/usr/local/bin/../Cellar/python3/3.6.3/bin/../Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py", line 207, in __subclasscheck__
    ok = cls.__subclasshook__(subclass)
  File "/private/tmp/tmp2/celery/.eggs/vine-1.1.4-py3.6.egg/vine/abstract.py", line 33, in __subclasshook__
    if cls is Thenable:
NameError: name 'Thenable' is not defined

Not sure if this is a MonkeyType issue, or a typing module one.

I have searched many pages and read MokeyType's Doc, such as:

https://engineering.instagram.com/let-your-code-type-hint-itself-introducing-open-source-monkeytype-a855c7284881
http://monkeytype.readthedocs.io/en/latest/faq.html
But I still don't know how to type-checking in Python. Anyone tell me? Or give a link for documents thanks

To make the simplest cases easier, instead of having to modify code to apply the monkeytype context manager. Should work similarly to coverage run: run the given Python script with monkeytype context manager applied.

Use Case: For projects with many tests and/or long-running tests it can take an extreme amount of time to generate all the traces serially. Pytest-xdist can be used to run tests in parallel processes. If there was a way to hook the tracer in to all the tests, that would really speed up the process.

Possible implementation; pytest-monkeytest plugin? Support enabling tracing in conftest.py?

Hi,

I'm getting errors like:
Module 'module.factory' has no attribute 'Class1_Class2' where the factory module is binding a custom class inherited from Class1 and Class2.

All of the traces are for the 'Class1_Class2' custom class, rather than the base classes it inherits from, so the traces don't match up.

Any ideas or suggestions on how to make monkeytype work with derived classes like this?

Thanks,
John

Would you be interested in an option like git diff so that users can know the differences between existing annotations and stubs generated only from traces?

This option should only be a sub-option to stub I think and can make use of --ignore-existing-annotations

Expected output: showing existing annotations and stubs learned only from traces (--ignore-existing-annotations) alternatively only for functions that disagree between the twos.

Potential benefits: quickly know where differences are.

Because visitor methods are defined like rewrite_Dict and called based on __name__ of the type, they can be called for unexpected types if there is a name collision. E.g. we have a class named Dict that is not typing.Dict, and when found in an annotation it results in this crash:

  File "/.../lib/python3.6/site-packages/monkeytype/stubs.py", line 574, in build_module_stubs_from_traces 
    defn = get_updated_definition(func, traces, rewriter=rewriter) 
  File "/.../lib/python3.6/site-packages/monkeytype/stubs.py", line 248, in get_updated_definition 
    arg_types = {name: rewriter.rewrite(typ) for name, typ in arg_types.items()} 
  File "/.../lib/python3.6/site-packages/monkeytype/stubs.py", line 248, in <dictcomp> 
    arg_types = {name: rewriter.rewrite(typ) for name, typ in arg_types.items()} 
  File "/.../python3.6/site-packages/monkeytype/typing.py", line 245, in rewrite 
    typ = rw.rewrite(typ) 
  File "/.../lib/python3.6/site-packages/monkeytype/typing.py", line 157, in rewrite 
    return rewriter(typ) 
  File "/.../lib/python3.6/site-packages/monkeytype/typing.py", line 129, in rewrite_Dict 
    return self._rewrite_container(Dict, dct) 
  File "/.../lib/python3.6/site-packages/monkeytype/typing.py", line 123, in _rewrite_container 
    if container.__args__ is None: 
AttributeError: type object 'Dict' has no attribute '__args__' 

instead of e.g. _GatheringFuture

Hey guys,

Excited to get this working, but can't seem to get this to find any traces.

The project in question has nothing to do with Django/webdev -- it is a library of numerically oriented classes and their calculations.

I haven't set a config, and am running this from CMD on a private package. The only "scripts" in the package are tests found as tests/[test].py

I'll try:

monkeytype run tests/[test].py

then

monkeytype stub tests/[test].py

or

monkeytype stub [libraryfile].py

And all I seem to ever get in return is No traces found.

Am I doing something wrong here? Are there certain assumptions regarding package setup that I'm missing?

Thanks.

Forward references in generated stubs should be quoted.

Running monkeytype stub or monkeytype apply on an module executes that module.

This is probably not easily preventable, but I do consider it unexpected behaviour, so it might be best to mention that in the docs somewhere.

Add support for listing the unique set of modules for which we have traces.

Sample usage:

> monkeytype list-modules
bar
baz
jaz

I'd like to be able to run monkeytype stub --all or something similar to generate all stubs that were recorded.

Trying with https://github.com/domoritz/draco.

monkeytype run draco/cli.py examples/histogram.json

Then monkeytype apply draco.spec results in

Traceback (most recent call last):
  File "/usr/local/anaconda3/envs/asp/bin/monkeytype", line 11, in <module>
    sys.exit(entry_point_main())
  File "/usr/local/anaconda3/envs/asp/lib/python3.6/site-packages/monkeytype/cli.py", line 257, in entry_point_main
    sys.exit(main(sys.argv[1:], sys.stdout, sys.stderr))
  File "/usr/local/anaconda3/envs/asp/lib/python3.6/site-packages/monkeytype/cli.py", line 245, in main
    handler(args, stdout, stderr)
  File "/usr/local/anaconda3/envs/asp/lib/python3.6/site-packages/monkeytype/cli.py", line 113, in apply_stub_handler
    subprocess.run(cmd, shell=True, check=True, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
  File "/usr/local/anaconda3/envs/asp/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'retype --pyi-dir /var/folders/g1/bvh0x71s0tg04jq6py_svvdh0000gp/T/monkeytype97qemy3r --target-dir /Users/domoritz/Developer/UW/draco/draco /Users/domoritz/Developer/UW/draco/draco/spec.py' returned non-zero exit status 1.

When collecting traces, we catch unexpected exceptions and just log them and continue execution: https://github.com/Instagram/MonkeyType/blob/master/monkeytype/tracing.py#L254

This is good, because there are lots of strange edge cases where trace collection could fail, and we don't want them to break the MonkeyType run entirely.

But it turns out there are also strange edge cases in trace serialization that can fail (e.g. see #33). It would be good if we were also robust against trace serialization failure; one weird trace failing to serialize shouldn't mean we lose all of our traces.

Given our current logger/store design, it looks like sadly this is something that has to be implemented independently by every trace store, because it's the trace store that is responsible for iterating over a list of traces, serializing them, and storing them.

We could provide a documented serialize_traces utility that takes Iterable[CallTrace] and returns Iterable[CallTraceRow] (or just yields rows, to avoid extra materialization of long lists), and we could implement the exception-catch-log-and-continue there, and then we could use this in the default SQLite store, and it'd be easy for third-party stores to reuse.

If monkeytype could be activated before every test run we'd be able to collect types more easily and efficiently.

For one of my modules, I ran test cases via monketype run, I can see the traces in the sqlite3 database, and I can run 'monketype stub modulename' and see what looks like valid stub output.

However, when I run 'monkeytype apply modulename', it returns no errors but the module is not modified.

I've been able to successfully update other modules. This module is my only good repro so far. There are no errors anywhere. I'm open to ideas on how to proceed. I guess i can start with debugging monkeytype?

E.g. if only a small percentage of production calls are traced, we may still want to use this to get more samples of a rarely-called function.

We should be able to detect this with a heuristic like "no traces found and os.path.exists(...)"

Shouldn't just use a naive endswith('.py'), it's perfectly valid to have a package or module named py.

Running monkeytype against a file with Django models or dependencies on Django apps gives me a series of errors.

First, just monkeytype apply package.app.models:

django.core.exceptions.ImproperlyConfigured: Requested setting DEFAULT_INDEX_TABLESPACE, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

Second, DJANGO_SETTINGS_MODULE=config.settings.local monkeytype apply package.app.models:

File "package/app/models.py"
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

Third, adding

import django
django.setup()

to the top of package.app.models:

File "some/other/models.py"
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

What has worked for me is patching monkeytype/cli.py and monkeytype/util.py to add

+   import django
+   django.setup()
    mod = importlib.import_module(module)

Obviously this isn't suitable as-is because it breaks monkeytype on non-Django codebases. However, it did give me the idea to have an ModuleImporter (name WIP) context manager that returns importlib.import_module(module) but may also perform pre-/post-import actions. It would work similarly to TypeRewriter: the default implementation could just import_module, but a DjangoModuleImporter could call django.setup(), and the specific ModuleImporter could be configured in settings.

Thoughts? Happy to submit a PR for this if you think it's the right direction.

Just saw NoneType hints generated on a method. I thought this was fixed.

I'll set up a repro.

e.g.

@staticmethod
    def _get_templates(device_name: str, device_templating: Dict[str, Dict[str, Union[str, NoneType]]], jinja_env: Environment, build_info: OrderedDict, step: str) -> Union[Tuple[Template, Template], Tuple[Template, NoneType]]: