instagram / monkeytype Goto Github PK
View Code? Open in Web Editor NEWA Python library that generates static type annotations by collecting runtime types
License: Other
A Python library that generates static type annotations by collecting runtime types
License: Other
Right now it just annotates those as OrderedDict
, instead of specifying the keys and values types like it does with regular dicts
To make the simplest cases easier, instead of having to modify code to apply the monkeytype context manager. Should work similarly to coverage run
: run the given Python script with monkeytype context manager applied.
Trying to set up dev environment per instructions in CONTRIBUTING.rst
Getting an import error:
Traceback (most recent call last):
File "/home/johnar/github/MonkeyType/monkeytype/cli.py", line 17, in
from typing import (
File "/home/johnar/github/MonkeyType/monkeytype/typing.py", line 10, in
from typing import (
ImportError: cannot import name 'Any'
In pycharm it actually shows that import _Any and _Union result in unresolved reference, as they're not defined in the typing module.
This is with stdlib typing 3.6.4:
typing 3.6.4
I'd like to be able to run monkeytype stub --all
or something similar to generate all stubs that were recorded.
When collecting traces, we catch unexpected exceptions and just log them and continue execution: https://github.com/Instagram/MonkeyType/blob/master/monkeytype/tracing.py#L254
This is good, because there are lots of strange edge cases where trace collection could fail, and we don't want them to break the MonkeyType run entirely.
But it turns out there are also strange edge cases in trace serialization that can fail (e.g. see #33). It would be good if we were also robust against trace serialization failure; one weird trace failing to serialize shouldn't mean we lose all of our traces.
Given our current logger/store design, it looks like sadly this is something that has to be implemented independently by every trace store, because it's the trace store that is responsible for iterating over a list of traces, serializing them, and storing them.
We could provide a documented serialize_traces
utility that takes Iterable[CallTrace]
and returns Iterable[CallTraceRow]
(or just yields rows, to avoid extra materialization of long lists), and we could implement the exception-catch-log-and-continue there, and then we could use this in the default SQLite store, and it'd be easy for third-party stores to reuse.
I've seen this #25 and tried to invoke pytest
command as monkeytype run pytest
but this results in following error
Traceback (most recent call last):
File "/home/user/.local/share/virtualenvs/venv/bin/monkeytype", line 11, in <module>
sys.exit(entry_point_main())
File "/home/user/.local/share/virtualenvs/venv/lib/python3.6/site-packages/monkeytype/cli.py", line 257, in entry_point_main
sys.exit(main(sys.argv[1:], sys.stdout, sys.stderr))
File "/home/user/.local/share/virtualenvs/venv/lib/python3.6/site-packages/monkeytype/cli.py", line 245, in main
handler(args, stdout, stderr)
File "/home/user/.local/share/virtualenvs/venv/lib/python3.6/site-packages/monkeytype/cli.py", line 130, in run_handler
runpy.run_path(args.script_path, run_name='__main__')
File "/usr/lib64/python3.6/runpy.py", line 261, in run_path
code, fname = _get_code_from_file(run_name, path_name)
File "/usr/lib64/python3.6/runpy.py", line 231, in _get_code_from_file
with open(fname, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'pytest'
Please follow your own practices espoused in https://code.facebook.com/posts/300798627056246/relicensing-react-jest-flow-and-immutable-js/
Re-license me maybe?
Because visitor methods are defined like rewrite_Dict
and called based on __name__
of the type, they can be called for unexpected types if there is a name collision. E.g. we have a class named Dict
that is not typing.Dict
, and when found in an annotation it results in this crash:
File "/.../lib/python3.6/site-packages/monkeytype/stubs.py", line 574, in build_module_stubs_from_traces
defn = get_updated_definition(func, traces, rewriter=rewriter)
File "/.../lib/python3.6/site-packages/monkeytype/stubs.py", line 248, in get_updated_definition
arg_types = {name: rewriter.rewrite(typ) for name, typ in arg_types.items()}
File "/.../lib/python3.6/site-packages/monkeytype/stubs.py", line 248, in <dictcomp>
arg_types = {name: rewriter.rewrite(typ) for name, typ in arg_types.items()}
File "/.../python3.6/site-packages/monkeytype/typing.py", line 245, in rewrite
typ = rw.rewrite(typ)
File "/.../lib/python3.6/site-packages/monkeytype/typing.py", line 157, in rewrite
return rewriter(typ)
File "/.../lib/python3.6/site-packages/monkeytype/typing.py", line 129, in rewrite_Dict
return self._rewrite_container(Dict, dct)
File "/.../lib/python3.6/site-packages/monkeytype/typing.py", line 123, in _rewrite_container
if container.__args__ is None:
AttributeError: type object 'Dict' has no attribute '__args__'
The end of the traceback looks like:
File "/.../lib/python3.6/site-packages/monkeytype/typing.py", line 180, in rewrite_Union
return Union[elems]
File "/.../lib/python3.6/typing.py", line 682, in inner
return func(*args, **kwds)
File "/.../lib/python3.6/typing.py", line 793, in __getitem__
raise TypeError("Cannot take a Union of no types.")
TypeError: Cannot take a Union of no types.
Haven't worked on a repro, but from looking at the code I think it ought to be pretty easy to repro with e.g. a List
parameter that is only ever called with an empty list. In this case we should just leave the annotation as List[Any]
, there's nothing else we can do.
I want to run
myscript update
but monkeytype run myscript update
errors with:
monkeytype: error: unrecognized arguments: update
Hey guys,
Excited to get this working, but can't seem to get this to find any traces.
The project in question has nothing to do with Django/webdev -- it is a library of numerically oriented classes and their calculations.
I haven't set a config, and am running this from CMD on a private package. The only "scripts" in the package are tests found as tests/[test].py
I'll try:
monkeytype run tests/[test].py
then
monkeytype stub tests/[test].py
or
monkeytype stub [libraryfile].py
And all I seem to ever get in return is No traces found.
Am I doing something wrong here? Are there certain assumptions regarding package setup that I'm missing?
Thanks.
By default MonkeyType respects existing annotations as the authority, and uses them over annotations inferred from traces when generating stubs.
For checking correctness of existing annotations against recorded traces, it would be useful to have an option to generate a stub that only considers the traces and ignores existing annotations.
Currently we generate e.g. WARNING: Failed decoding trace for function ...: Module 'django.utils.functional' has no attribute 'lazy.<locals>'
, but it should be possible to special-case this and extract the contained type.
Trying with https://github.com/domoritz/draco.
monkeytype run draco/cli.py examples/histogram.json
Then monkeytype apply draco.spec
results in
Traceback (most recent call last):
File "/usr/local/anaconda3/envs/asp/bin/monkeytype", line 11, in <module>
sys.exit(entry_point_main())
File "/usr/local/anaconda3/envs/asp/lib/python3.6/site-packages/monkeytype/cli.py", line 257, in entry_point_main
sys.exit(main(sys.argv[1:], sys.stdout, sys.stderr))
File "/usr/local/anaconda3/envs/asp/lib/python3.6/site-packages/monkeytype/cli.py", line 245, in main
handler(args, stdout, stderr)
File "/usr/local/anaconda3/envs/asp/lib/python3.6/site-packages/monkeytype/cli.py", line 113, in apply_stub_handler
subprocess.run(cmd, shell=True, check=True, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
File "/usr/local/anaconda3/envs/asp/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'retype --pyi-dir /var/folders/g1/bvh0x71s0tg04jq6py_svvdh0000gp/T/monkeytype97qemy3r --target-dir /Users/domoritz/Developer/UW/draco/draco /Users/domoritz/Developer/UW/draco/draco/spec.py' returned non-zero exit status 1.
If monkeytype could be activated before every test run we'd be able to collect types more easily and efficiently.
I've run monkeytype run manage.py test
in my django project after pip install monkeytype
(inside of a virtual env) and I'm not seeing any output of the sqllite file described in the README.
Running monkeytype stub
for any of my modules doesn't work either. If there's something different that I have to do for django tests, let's add that to the README. Otherwise, am I missing something?
E.g. someone should be able to implement support for django.utils.functional.cached_property
via config, we shouldn't need to hardcode support in core.
Running monkeytype against a file with Django models or dependencies on Django apps gives me a series of errors.
First, just monkeytype apply package.app.models
:
django.core.exceptions.ImproperlyConfigured: Requested setting DEFAULT_INDEX_TABLESPACE, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
Second, DJANGO_SETTINGS_MODULE=config.settings.local monkeytype apply package.app.models
:
File "package/app/models.py"
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
Third, adding
import django
django.setup()
to the top of package.app.models:
File "some/other/models.py"
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
What has worked for me is patching monkeytype/cli.py
and monkeytype/util.py
to add
+ import django
+ django.setup()
mod = importlib.import_module(module)
Obviously this isn't suitable as-is because it breaks monkeytype on non-Django codebases. However, it did give me the idea to have an ModuleImporter
(name WIP) context manager that returns importlib.import_module(module)
but may also perform pre-/post-import actions. It would work similarly to TypeRewriter: the default implementation could just import_module
, but a DjangoModuleImporter could call django.setup()
, and the specific ModuleImporter could be configured in settings.
Thoughts? Happy to submit a PR for this if you think it's the right direction.
See #48 for a reproducible example case. Running retype directly shows a useful error, but somehow this doesn't get passed through to MonkeyType output, even though we pipe both stdout and stderr in the subprocess call. Need to investigate why not.
E.g. if only a small percentage of production calls are traced, we may still want to use this to get more samples of a rarely-called function.
Report the number of samples that were used to generate a method's stub. This will help gauge confidence in the results.
We should be able to detect this with a heuristic like "no traces found and os.path.exists(...)"
Shouldn't just use a naive endswith('.py')
, it's perfectly valid to have a package or module named py
.
Just saw NoneType hints generated on a method. I thought this was fixed.
I'll set up a repro.
e.g.
@staticmethod
def _get_templates(device_name: str, device_templating: Dict[str, Dict[str, Union[str, NoneType]]], jinja_env: Environment, build_info: OrderedDict, step: str) -> Union[Tuple[Template, Template], Tuple[Template, NoneType]]:
Running monkeytype stub
or monkeytype apply
on an module executes that module.
This is probably not easily preventable, but I do consider it unexpected behaviour, so it might be best to mention that in the docs somewhere.
Thx for this awesome project!
This was the code which threw the error:
def get_types():
class Type(object):
def __init__(self, name, slug):
self.name = name
self.slug = slug
return [Type(type_,type_.lower())
for type_ in get_types_from_somewhere_else()]
https://www.python.org/dev/peps/pep-0498/
def a(b):
return f'b == {b}'
Currently, monkeytype errors on apply with
error: .../monkey/test.py: Cannot parse: 2:12: return f'b == {b}'
git clone https://github.com/celery/celery
cd celery/
python setup.py test # for quick install of test dependencies
monkeytype run setup.py test
Failed collecting trace
Traceback (most recent call last):
File "/private/tmp/tmp2/celery/venv/lib/python3.6/site-packages/monkeytype/tracing.py", line 252, in __call__
self.handle_return(frame, arg)
File "/private/tmp/tmp2/celery/venv/lib/python3.6/site-packages/monkeytype/tracing.py", line 227, in handle_return
typ = get_type(arg)
File "/private/tmp/tmp2/celery/venv/lib/python3.6/site-packages/monkeytype/typing.py", line 76, in get_type
val_type = shrink_types(get_type(v) for v in obj.values())
File "/private/tmp/tmp2/celery/venv/lib/python3.6/site-packages/monkeytype/typing.py", line 39, in shrink_types
return Union[types]
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/typing.py", line 679, in inner
return cached(*args, **kwds)
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/typing.py", line 803, in __getitem__
return self.__class__(parameters, origin=self, _root=True)
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/typing.py", line 743, in __new__
parameters = _remove_dups_flatten(parameters)
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/typing.py", line 647, in _remove_dups_flatten
for t2 in all_params - {t1}
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/typing.py", line 649, in <genexpr>
t2.__origin__ is not None)):
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/typing.py", line 1153, in __subclasscheck__
return super().__subclasscheck__(cls)
File "/usr/local/bin/../Cellar/python3/3.6.3/bin/../Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py", line 207, in __subclasscheck__
ok = cls.__subclasshook__(subclass)
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/typing.py", line 884, in __extrahook__
if issubclass(subclass, scls):
File "/usr/local/bin/../Cellar/python3/3.6.3/bin/../Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py", line 207, in __subclasscheck__
ok = cls.__subclasshook__(subclass)
File "/private/tmp/tmp2/celery/.eggs/vine-1.1.4-py3.6.egg/vine/abstract.py", line 33, in __subclasshook__
if cls is Thenable:
NameError: name 'Thenable' is not defined
Not sure if this is a MonkeyType issue, or a typing module one.
I want to add types to myscript.py
,
so I do monkeytype run myscript.py
but monkeytype stub myscript.py
says No traces found
so I dumped monkeytype.sqlite3
and see it has traces under the module named __main__
,
but monkeytype stub __main__
says ERROR: Failed decoding trace: Module '__main__' has no attribute 'main'
The newer version of MonkeyType does not work in IPython. I think e8bb8a6 could be the reason.
An interactive session in IPython will assert __name__ == '__main__'
it would still be meaningful to capture the CallTrace
so folks could interactively type their notebook/python source.
I can totally understand if this falls outside the scope of the project.
The stubs generated by monkeytype don't preserve annotations that use type aliases. For example, if a function has the following signature:
Number = Union[int, float]
def sum(a: Number, b: Number) -> Number:
return a + b
MonkeyType will expand Number. The generated stub will be:
def sum(a: Union[int, float], b: Union[int, float]) -> Union[int, float]: ...
Attempting to apply a stub like this will fail with an error from retype of the form incompatible annotation for ...
.
I'm not sure if there is anything we can do about this. One work around would be an option to omit functions that already have annotation from the generated stubs.
Everything I try to do with the monkeytype
command-line took gives me this:
Traceback (most recent call last):
File "/Users/guido/v36/bin/monkeytype", line 7, in <module>
from monkeytype.cli import entry_point_main
File "/Users/guido/v36/lib/python3.6/site-packages/monkeytype/__init__.py", line 12, in <module>
from monkeytype.config import (
File "/Users/guido/v36/lib/python3.6/site-packages/monkeytype/config.py", line 83, in <module>
venv_real_prefix = getattr(sys, 'real_prefix')
AttributeError: module 'sys' has no attribute 'real_prefix'
This is with a venv running Python 3.6.2 on Mac. Maybe you meant getattr(sys, 'real_prefix', None)
?
After running monkeytype.exe run on my script (Windows 10, python 3.6.2), I get the error below:
Notice that the .sql file is dumped, and the error only occurs after my script is executed.
ERROR:monkeytype.tracing:Failed collecting trace
Traceback (most recent call last):
File "c:\program files\python36\lib\site-packages\monkeytype\tracing.py", li ne 252, in call
self.handle_return(frame, arg)
File "c:\program files\python36\lib\site-packages\monkeytype\tracing.py", li ne 227, in handle_return
typ = get_type(arg)
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 76, in get_type
val_type = shrink_types(get_type(v) for v in obj.values())
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 34, in shrink_types
types = tuple(types)
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 76, in
val_type = shrink_types(get_type(v) for v in obj.values())
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 62, in get_type
return Type[obj]
File "c:\program files\python36\lib\typing.py", line 652, in inner
return func(*args, **kwds)
File "c:\program files\python36\lib\typing.py", line 1098, in getitem
params = tuple(_type_check(p, msg) for p in params)
File "c:\program files\python36\lib\typing.py", line 1098, in
params = tuple(_type_check(p, msg) for p in params)
File "c:\program files\python36\lib\typing.py", line 376, in _type_check
raise TypeError("Plain %s is not valid as type argument" % arg)
TypeError: Plain typing.Generic is not valid as type argument
ERROR:monkeytype.tracing:Failed collecting trace
Traceback (most recent call last):
File "c:\program files\python36\lib\site-packages\monkeytype\tracing.py", li ne 252, in call
self.handle_return(frame, arg)
File "c:\program files\python36\lib\site-packages\monkeytype\tracing.py", li ne 227, in handle_return
typ = get_type(arg)
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 76, in get_type
val_type = shrink_types(get_type(v) for v in obj.values())
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 34, in shrink_types
types = tuple(types)
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 76, in
val_type = shrink_types(get_type(v) for v in obj.values())
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 62, in get_type
return Type[obj]
File "c:\program files\python36\lib\typing.py", line 652, in inner
return func(*args, **kwds)
File "c:\program files\python36\lib\typing.py", line 1098, in getitem
params = tuple(_type_check(p, msg) for p in params)
File "c:\program files\python36\lib\typing.py", line 1098, in
params = tuple(_type_check(p, msg) for p in params)
File "c:\program files\python36\lib\typing.py", line 376, in _type_check
raise TypeError("Plain %s is not valid as type argument" % arg)
TypeError: Plain typing.Generic is not valid as type argument
ERROR:monkeytype.tracing:Failed collecting trace
Traceback (most recent call last):
File "c:\program files\python36\lib\site-packages\monkeytype\tracing.py", li ne 252, in call
self.handle_return(frame, arg)
File "c:\program files\python36\lib\site-packages\monkeytype\tracing.py", li ne 227, in handle_return
typ = get_type(arg)
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 76, in get_type
val_type = shrink_types(get_type(v) for v in obj.values())
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 34, in shrink_types
types = tuple(types)
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 76, in
val_type = shrink_types(get_type(v) for v in obj.values())
File "c:\program files\python36\lib\site-packages\monkeytype\typing.py", lin e 62, in get_type
return Type[obj]
File "c:\program files\python36\lib\typing.py", line 652, in inner
return func(*args, **kwds)
File "c:\program files\python36\lib\typing.py", line 1098, in getitem
params = tuple(_type_check(p, msg) for p in params)
File "c:\program files\python36\lib\typing.py", line 1098, in
params = tuple(_type_check(p, msg) for p in params)
File "c:\program files\python36\lib\typing.py", line 376, in _type_check
raise TypeError("Plain %s is not valid as type argument" % arg)
TypeError: Plain typing.Generic is not valid as type argument
Storing only unique traces in the logger has a couple advantages:
There doesn't seem to be much use case for seeing the actual distribution of traces, but if someone needs that, it's only a few lines of code to implement their own logger.
Given
UserId = NewType('UserId', int)
def get_user(self, user_id: UserId) -> 'User':
MonkeyType generates
def get_user(
self,
user_id: <function NewType.<locals>.new_type at 0x1090a52f0>
) -> 'User': ...
which then causes retype to fail with this error message:
error: /path/to/module.py: invalid syntax (<unknown>, line 18)
This happens even when --exclude-unparsable-defaults
is passed.
I was able to come up with a solution to this, which I'll submit a PR for. However, it seems hacky to me and I'd appreciate feedback on it.
For one of my modules, I ran test cases via monketype run, I can see the traces in the sqlite3 database, and I can run 'monketype stub modulename' and see what looks like valid stub output.
However, when I run 'monkeytype apply modulename', it returns no errors but the module is not modified.
I've been able to successfully update other modules. This module is my only good repro so far. There are no errors anywhere. I'm open to ideas on how to proceed. I guess i can start with debugging monkeytype?
$ monkeytype run
Traceback (most recent call last):
File "/usr/local/bin/monkeytype", line 7, in
from monkeytype.cli import entry_point_main
File "/usr/local/lib/python3.5/dist-packages/monkeytype/init.py", line 7, in
from typing import (
ImportError: cannot import name 'ContextManager'
Forward references in generated stubs should be quoted.
Add support for listing the unique set of modules for which we have traces.
Sample usage:
> monkeytype list-modules
bar
baz
jaz
We currently can't rehydrate builtins.traceback.
I have searched many pages and read MokeyType's Doc, such as:
https://engineering.instagram.com/let-your-code-type-hint-itself-introducing-open-source-monkeytype-a855c7284881
http://monkeytype.readthedocs.io/en/latest/faq.html
But I still don't know how to type-checking in Python. Anyone tell me? Or give a link for documents thanks
Use Case: For projects with many tests and/or long-running tests it can take an extreme amount of time to generate all the traces serially. Pytest-xdist can be used to run tests in parallel processes. If there was a way to hook the tracer in to all the tests, that would really speed up the process.
Possible implementation; pytest-monkeytest plugin? Support enabling tracing in conftest.py?
instead of e.g. _GatheringFuture
e.g. should not generate from _io import BytesIO
in a stub
Hi,
I'm getting errors like:
Module 'module.factory' has no attribute 'Class1_Class2' where the factory module is binding a custom class inherited from Class1 and Class2.
All of the traces are for the 'Class1_Class2' custom class, rather than the base classes it inherits from, so the traces don't match up.
Any ideas or suggestions on how to make monkeytype work with derived classes like this?
Thanks,
John
Would you be interested in an option like git diff
so that users can know the differences between existing annotations and stubs generated only from traces?
This option should only be a sub-option to stub
I think and can make use of --ignore-existing-annotations
Expected output: showing existing annotations and stubs learned only from traces (--ignore-existing-annotations) alternatively only for functions that disagree between the twos.
Potential benefits: quickly know where differences are.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.