Utility to validate stubfile signatures against .py implementations about pytypes HOT 4 OPEN

Besides validating, there could be a fixer tool to add missing definitions to the stubfile, and if there are incompatible definitions I'd expect both tools to notice and warn you.

from pytypes.

Note on the use case:
Advertising this mainly for Python 3.4 feels somewhat disappointing or like an understatement at least.
I think there are use cases for stubfiles way beyond Python 3.4 compatibility. (Isn't, Python 3.4 EOL anyway?)
C-extensions and typeshed are other use cases and some people still feel that type annotations pollute Python code and thus prefer stubfiles.

A starting point may be https://github.com/Stewori/pytypes/blob/master/pytypes/stubfile_manager.py#L55
It recursively walks nested classes and I think it would cause an attribute error if the pyi contains classes that are missing in the py. It would have to be extended (i.e. under new name and without inserting the ._match_type) to check for other members and compare function/method signatures. Finally a roof function would call it twice, i.e. additionally with the roles of py and pyi swapped to assert totality.

Note:
The route to do this based on inspection might not be the most desirable one. As it has to import both modules (py and pyi) it is not suitable for untrusted code; but then Python isn't in general. A safer approach would be based on the AST module. However the inspection approach is much easier and closer to the current spirit of pytypes. For some reason I once decided to go the inspection route and literally everything in pytypes works inspection-based. So I'd suggest not to change this now. And btw it has the advantage that it also works with C-extension modules seamlessly.

from pytypes.

That's true stubfiles aren't just for py 3.4 compatibility. I thought C-extensions wouldn't be feasible to cross-check and typeshed seemed "legacy" per comments like python/typeshed#3991 (comment). Anyway, sorry if I undersold the value.

Actually mypy does have a stubtest tool that seems related, but it doesn't seem to complain if I delete things from pyi or mess up signatures.

from pytypes.

I thought C-extensions wouldn't be feasible to cross-check

I think that should work based on inspection just like ordinary python modules, except that type annotations are only possible via stubfiles. But disclaimer: I did not actually test this so far.

typeshed seemed "legacy"

Honestly, I did not follow the fate of typeshed recently but I just had a quick look at their landing page and there is no deprecation statement whatsoever. I learned that mypy bundles typeshed, so it is much invisible to the user though. (Maybe I or we should consider how to improve typeshed integration for pytypes but that's again a different story.)

Actually mypy does have a stubtest tool that seems related

If you already figured it out, would you paste a usage instruction or ideally a small script that applies it to the upcoming pytypes' .pyi files? This may be a trivial thing but all these tiny things add up, and having it as a script to make it a true nobrainer helps to save some time...

from pytypes.