Comments (6)
Hi @henrylhtsang.
Unfortunately, there's an architectural issue in Pyre (and a number of other tools too) where we assume that a fully qualified name means something unambiguous in Python, so that torchrec.distributed.shard
has to refer to either a module or a function but not both.
The pattern of shadowing a module with a function imported into the parent then causes any tool indexing the code this way to fail to understand it. Until recently, Pyre was failing to analyze the code (which meant it dropped type information) silently, and we recently changed Pyre to warn users that the import can't be analyzed.
This is pretty disruptive, and we also hope we can change Pyre to understand that fully qualified names are not unambiguous and it's necessary to model the effect of an import on scope more precisely, but this will take at least several months for us to fix, possibly longer.
In the meantime, your best bet is probably to rename something. Because you are a library and backward compatibility matters, it's tricky to decide what exactly to do, but:
- if users typically run
from torchrec.distributed import shard
then you probably don't want to change the meaning of that, which means you might want to changeshard.py
tosharding.py
or something like that - if you're worried that users are also directly importing the module (
import torchrec.distributed.shard
, orfrom torchrec.distributed.shard import <something>
, then you could potentially also keepshard.py
and modify it to justfrom .sharding import *
. At that point, Pyre won't be able to seeshard.py
but at least at runtime it will be there, so users won't see breakages (unless they are using Pyre)
from pyre-check.
Totally understand the frustration. This is a known issue. Unfortunately, the fix will take some time to land, so I would recommend finding a workaround.
from pyre-check.
@migeed-z thanks, is there a timeline?
from pyre-check.
(You can of course use a # pyre-ignore
to suppress the error you are seeing, but unfortunately users of your library who are using Pyre - which includes most Meta users - will run into the same problem in that case so I wouldn't recommend that approach)
from pyre-check.
@stroxler thanks, will explore changing names
from pyre-check.
This is a duplicate of #232
from pyre-check.
Related Issues (20)
- Update pyre-check.org/play to use Python 3.11 HOT 2
- [BUG] Check command exited with non-zero return code: 126. HOT 1
- CI builds are broken HOT 1
- Pyre / Pysa unable to analyse files with same name in different folders HOT 2
- Taint config on "Running Pysa" page is invalid HOT 1
- `reveal_type` does not work reliably HOT 3
- Unable to get SQLAlchemy sinks to work HOT 4
- Pyre failing in python 3.8 and 3.9 due to new syntax HOT 10
- `input` is not part of the environment, no module `input` in search path. HOT 2
- Cannot resolve type for pymysql HOT 2
- Pyre command hangs on on torch.ao.ns.fx.mappings.get_node_type_to_io_type_map HOT 2
- [pysa][MLH] Add a high level API for step logging
- Pyre returning ERROR Check command exited with non-zero return code: 1 when running with poetry HOT 17
- How to support env['PYTHONPATH'] HOT 1
- How to use OS.enviorn as a taint source HOT 2
- Incremental check hangs on OS X HOT 3
- [pysa][MLH] Add a show to Registry
- [pysa][MLH] Hide warnings about high number of overrides
- [pysa][MLH] Hide model verification error for "X is not part of the environment"
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyre-check.