Comments (9)
Opening the first python file of a project takes a little longer because it needs to populate the cache, but once the cache is populated, the 2 functions should run substantially faster. Is this not the case for you?
from emacs-pet.
Opening the first python file of a project takes a little longer because it needs to populate the cache, but once the cache is populated, the 2 functions should run substantially faster.
I would have also expected that, but it is not the case.
from emacs-pet.
I see this problem too - it took me a while to understand why opening of my Python files is so incredibly slow.
from emacs-pet.
It's expected to take a few seconds when opening the first python file in a project, but opening subsequent python files within the same project should be relatively quick due to the aggressive caching. The slowness mainly comes from the number of subprocesses Emacs has to open in order to parse the config and cache files. It's a problem if opening subsequent python files within the same project is slow, but I'll need more information such as your project layout and benchmark numbers to even confirm it is indeed a bug. There are opportunities to speed up opening even the first file, but it's not worth the effort. Emacs 29 will come with a number of improvements that will enable this.
from emacs-pet.
I see this in Windows 10 where Emacs is generally extremely slugish (had to turn off pet, lsp-mode and a few other things to even get to a point that I don't need to wait 30 seconds to open each file ...), so I guess you can disregard my comment. In Linux/WSL it works fine - there is a slight delay when pet is enabled in the python-mode-hook but nothing dramatic.
from emacs-pet.
OK, I did some more testing - in Linux, under WSL but all files are on the Linux filesystem (not accessing NTFS, that would be very slow).
I have multiple files from the same project open already, my hook is starting pet
and looking for several executables when I open another Python file from the same project. This is what the profiler gives me (I have instrumented the entire pet
package):
Function Call count Elapsed Average elapsed
my-pet-hook 1 20.144855874 20.144855874
pet-executable-find 10 18.975925474 1.8975925474
pet-virtualenv-root 11 13.110769078 1.191888098
pet-flycheck-toggle-local-vars 1 9.248647607 9.248647607
pet-find-file-from-project 73 3.9797493269 0.0545171140
pet-find-file-from-project-root-recursively 33 3.504320876 0.1061915416
pet-use-pyenv-p 11 1.285627846 0.1168752587
pet-python-version 11 1.2855474310 0.1168679482
pet-python-version-path 11 1.2854736979 0.1168612452
pet-use-poetry-p 11 1.267356647 0.1152142406
pet-pyproject 11 1.267226448 0.1152024043
pet-pyproject-path 11 1.267143984 0.1151949076
pet-use-conda-p 11 1.2144122179 0.1104011107
pet-environment 11 1.2143436120 0.1103948738
pet-environment-path 11 1.2142735820 0.1103885074
pet-project-root 205 0.6063872709 0.0029579866
pet-find-file-from-project-root 73 0.2436195300 0.0033372538
pet-locate-dominating-file 55 0.228690813 0.0041580147
pet-pipfile-path 22 0.1521649430 0.0069165883
pet-use-pipenv-p 11 0.082937444 0.0075397676
pet-pipfile 11 0.0822894039 0.0074808549
pet-pre-commit-config 18 0.0613882240 0.0034104568
pet-pre-commit-config-path 18 0.0612433259 0.0034024069
pet-use-pre-commit-p 10 0.0349311759 0.0034931175
pet-pre-commit-config-has-hook-p 8 0.02727297 0.0034091212
pet-flycheck-python-pylint-find-pylintrc 1 0.005619078 0.005619078
pet-flycheck-checker-get-advice 170 0.0006075750 3.573...e-06
pet-flycheck-setup 1 0.000163408 0.000163408
pet-report-error 14 5.454...e-05 3.896...e-06
eglot-ensure 1 4.0053e-05 4.0053e-05
(my-pet-hook
is the top level hook that starts pet
, eglot-ensure
is there just to check it wasn't language server causing it, ignore those two)
It is clear that something isn't cached because those executables should be the same, regardless of which file from the project I open. I had 3 other Python files from that project open already - and it is this slow whenever I try to open another one. pet-virtualenv-root
seems to take ages for some reason.
For info:
- Pipenv project, Pipfile is in not in the project root but in a subdirectory, the source files being opened are in subfolders of that subdirectory with the Pipfile. Virtual environment is created in
.venv
folder next to the Pipfile. - We are using git and pre-commit for the hooks (yes, it is that project ...)
- Emacs is self-built from master but I have the same behavior with 28.x and 29 dailies.
- Pet is
7620c18 (HEAD -> main, tag: v1.1.0, origin/main, origin/HEAD) version 1.1.0
from emacs-pet.
Oof, OK I see why the problem happens for me. WSL automatically "helpfully" adds the content of Windows PATH variable to the Linux PATH. So when I didn't create the virtual environment for the current worktree yet, pet
started to search on the Windows filesystem for pipenv
and other binaries. Which is extremely slow.
Once I have created a local virtual environment, the file opening time dropped from 20 seconds to about 2s. Still a bit sluggish but much more manageable. This is the profile after the the cache has been populated:
my-pet-hook 1 2.163337651 2.163337651
pet-executable-find 10 2.1546226499 0.2154622649
pet-flycheck-toggle-local-vars 1 0.880342973 0.880342973
pet-project-root 29 0.07939641 0.0027378072
pet-pre-commit-config 18 0.0552564690 0.0030698038
pet-pre-commit-config-path 18 0.055118069 0.0030621149
pet-find-file-from-project 18 0.0549999620 0.0030555534
pet-find-file-from-project-root 18 0.0546787409 0.0030377078
pet-use-pre-commit-p 10 0.0317966749 0.0031796674
pet-virtualenv-root 11 0.028649633 0.0026045120
pet-pre-commit-config-has-hook-p 8 0.024069906 0.0030087382
pet-flycheck-python-pylint-find-pylintrc 1 0.005421203 0.005421203
pet-flycheck-checker-get-advice 173 0.0002698909 1.560...e-06
pet-flycheck-setup 1 0.000240067 0.000240067
eglot-ensure 1 2.3442e-05 2.3442e-05
pet-system-bin-dir 10 9.872...e-06 9.872e-07
pet-report-error 1 3.47e-06 3.47e-06
I am not sure whether it makes sense to add any sort of fix/workaround for this since this is a Windows/WSL specific nonsense and not really pet's fault but maybe mentioning this in the README would save some hair to some future user. Alternatively, if it detects poetry/pipenv but doesn't find a virtual environment, it could emit a warning message - "Hey, idiot, you forgot to run pipenv install
/poetry install
!"
However, caching the found executables (and not only the virtualenv root) per project would be likely still good idea that could make pet a good deal snappier. At 200ms per binary it is still 2 seconds delay when opening a file. Such cache would probably make it usable even in Windows once one "pays the price" for populating the cache with the first opened file.
from emacs-pet.
I'm going to close this. Please reopen if this is still an issue.
from emacs-pet.
This is still an issue on the latest commit for me. With pet-mode, (benchmark-run 1 (find-file "<some python file>"))
takes 0.6 seconds on average (not first python file opened for project). (pet-buffer-local-vars-setup)
takes ~0.15 seconds itself. Not sure if the flycheck setup is running multiple times, but some interaction between pet and lsp makes (lsp)
take 0.4s vs like 0.02 (using pyright and ruff-lsp).
I've switched from pet_mode
to using my own function that only sets the necessary variables using pet like the advanced configuration example in the readme, and it now takes <=0.1s to load a python file again.
from emacs-pet.
Related Issues (20)
- pet-find-file-in-project problem with regex in filename HOT 4
- Usage of string-match-p HOT 5
- Made list of changed variables configurable HOT 1
- `pet-exectuable-find` calls of `process-lines` using `pyenv which` can fail, setting vars to `nil` HOT 10
- Unable to guess virtualenv HOT 4
- Doesn't work on remotes over TRAMP HOT 2
- Add support for Eglot (and maybe Eshell) HOT 19
- pet-20230423.2326/pet.elc failed to define function pet HOT 7
- pyenv exited with status 127 when using pet-mode with emacs-jupyter HOT 5
- Support flymake? HOT 6
- pylsp-mypy is not recognizing the virtual environment HOT 3
- Virtualenv is not detected when it is directly in the root directory HOT 2
- Does conda work? HOT 4
- Manual venv selection? HOT 1
- Error in installation for newer emacs version HOT 5
- hatch support
- rye support
- pixi support
- Conflict between precommit and poetry HOT 15
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from emacs-pet.