bradenm / micropy-cli Goto Github PK

View Code? Open in Web Editor NEW

305.0 10.0 23.0 43.29 MB

Micropython Project Management Tool with VSCode support, Linting, Intellisense, Dependency Management, and more!

Home Page: https://micropy-cli.readthedocs.io

License: MIT License

Python 99.18% Makefile 0.75% Shell 0.08%

micropython cli automate stubs microcontroller vscode pylint intellisense autocompletion linting

micropy-cli's People

Contributors

Stargazers

Watchers

micropy-cli's Issues

Autocompletion and linting not available for libraries in src/lib

Hi there,

Problem

I am having some problems with libraries not getting used for auto completion/
Default sys.path is the root folder and /lib

>>> import sys
>>> sys.path
['', '/lib']

The resulting .vscode\settings.json and .pylintrc do not have this folder included.
Currently this results in not getting auto completion and proper linting for user defined libraries.

Expected behaviour:
Since the /lib folder on the device is available for imports, it should have working auto completion and linting working properly.

Possible fix:
Because user libraries go into the lib folder I propose adding the "src/lib" folder to the .vscode\settings.json and .pylintrc file.

What are your thoughts? Let me know!

Add support for Lego EV3 MicroPython

Add support for Lego EV3 MicroPython. https://education.lego.com/en-us/support/mindstorms-ev3/python-for-ev3

Error en install package Ultrasonic

When I try to install package Ultrasonic, I tog follow errors.

micropy install Ultrasonic

MicroPy Loading Project
MicroPy esp32-micropython-1.11.0 is already installed!

MicroPy ✔ Project Ready!

MicroPy Installing Packages
MicroPy Adding Ultrasonic to requirements...
Traceback (most recent call last):
File "c:\users\flemm\appdata\local\programs\python\python37\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\Flemm\AppData\Local\Programs\Python\Python37\Scripts\micropy.exe_main.py", line 9, in
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 764, in call
return self.main(*args, **kwargs)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 717, in main
rv = self.invoke(ctx)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 1137, in invoke
return process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 555, in invoke
return callback(*args, **kwargs)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\click\decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 555, in invoke
return callback(*args, **kwargs)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\micropy\cli.py", line 148, in install
project.add_package(pkg, dev=dev)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\micropy\project\project.py", line 241, in add_package
self.load_packages()
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\micropy\project\project.py", line 164, in load_packages
meta = utils.get_package_meta(name, spec=spec)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\micropy\utils\helpers.py", line 268, in get_package_meta
data = resp.json()
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\requests\models.py", line 897, in json
return complexjson.loads(self.text, **kwargs)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\json_init.py", line 348, in loads
return _default_decoder.decode(s)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "c:\users\flemm\appdata\local\programs\python\python37\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 7 column 1 (char 6)

Does this work with vscode-insider?

I've followed all the instruction provided, but the IntelliSense down't show up anything.
Is this because of the "insider version"?

Implement Remote stub source

micropy-stubs will eventually be an easy way to manage stubs from all different devices in a defined manor/structure. By using AWS S3, micropy-cli will use it as its primary source for stubs.

CreateStubs hangs on PyCom Gpy

Describe the bug

micropy stubs create /dev/tty.usbmodemPy06c88a1 --verbose

MicroPy  Connecting to Pyboard @ /dev/tty.usbmodemPy06c88a1
Connecting to /dev/tty.usbmodemPy06c88a1 (buffer-size 512)...
Trying to connect to REPL

Micropy never gets past this part... its just hung.

Logs

[2020-03-23 13:53:44] DEBUG: hookproxy(add_from_file): Method added to proxy: (<function PackagesModule.add_from_file at 0x10b715510>, '_hook__add_from_file__path_None__dev_False')
[2020-03-23 13:53:44] DEBUG: hookproxy(add_package): Method added to proxy: (<function PackagesModule.add_package at 0x10b715620>, '_hook__add_package__dev_False')
[2020-03-23 13:53:44] DEBUG: hookproxy(add_package): Method added to proxy: (<function DevPackagesModule.add_package at 0x10b715bf8>, '_hook__add_package__dev_True')
[2020-03-23 13:53:44] DEBUG: hookproxy(add_from_file): Method added to proxy: (<function DevPackagesModule.add_from_file at 0x10b715d08>, '_hook__add_from_file__path_None__dev_True')
[2020-03-23 13:53:44] DEBUG: hookproxy(stubs): Method added to proxy: (<function StubsModule.stubs at 0x10b747bf8>, '_hook__stubs__')
[2020-03-23 13:53:44] DEBUG: hookproxy(add_stub): Method added to proxy: (<function StubsModule.add_stub at 0x10b74a1e0>, '_hook__add_stub__')
[2020-03-23 13:53:44] DEBUG: micropy: MicroPy Loaded
[2020-03-23 13:53:44] INFO: micropy: Connecting to Pyboard @ /dev/tty.usbmodemPy06c88a1
[2020-03-23 13:53:44] DEBUG: pyboard: connecting to pydevice @ /dev/tty.usbmodemPy06c88a1

Context (Environment)

OS

MacOS: release='19.3.0', version='Darwin Kernel Version 19.3.0

Micropy Version:

micropy, version 3.3.0

Python Version

Python 3.6.10

VSCode Version

Version: 1.43.1
Commit: fe22a9645b44368865c0ba92e2fb881ff1afce94
Date: 2020-03-18T07:01:41.240Z
Electron: 7.1.11
Chrome: 78.0.3904.130
Node.js: 12.8.1
V8: 7.8.279.23-electron.0
OS: Darwin x64 19.3.0

Create Directory Junction instead making symbolic links on Windows

Junction (mklink /J) can be made without administrative privileges on Windows.

Update Python Versions

Before I submit a PR, wanted to run this by you.

Currently .python-version has
3.7.3
3.6.8

I proposed updating this to:
3.8.3
3.7.7
3.6.9

The Travis and GH-Actions in TOX.ini and would also need to be updated

[tox]
isolated_build = true
envlist = py38, py37, py36, flake8

[travis]
python =
    3.8: py38
    3.7: py37
    3.6: py36

[gh-actions]
python =
    3.8: py38
    3.7: py37
    3.6: py36

Not sure if any other places need updating for this OR if any other dependencies will need to be updated as well.

Use Symlinks for Stub Paths

Since VSCode doesn't allow environment variables in settings.json files, (see microsoft/vscode#2809), the generated settings.json and .pylintrc aren't very VCS friendly.

To get around this, I think it would be a good idea to generate a symlink (ignored by VCS) at the root of a new project that directs towards the users .micropy folder. This way, once #10 is finished, we can recreate this link when someone else clones the repo and initializes micropy, effectively making the pylint and vscode files VCS friendly.

AttributeError: 'TemplatesModule' object has no attribute 'provider'

Hi,
in the beginning, thank you for the work you're putting in here.
I work under Win10 in VS code with the esp32 and get the error message "AttributeError: 'TemplatesModule' object has no attribute 'provider'" when I try to create a new project. What can I do about it? I didn't find anything in the code spontaneously.

Project Dependencies

Handle Project dependencies by:

Tracking them
Bringing intellisense/autocompletion/linting to them
Managing requirements.txt files

Organize Tests by Module

Take some time to properly organize all the tests.

Probably would be best to split them up my module and giving them individual conftest.py(s) or something.

Add ability to init existing projects

Allow micropy init to work in existing projects

Add Micropy Stubs Testing Repo

Add optional Testing Repo for incomplete/unstable stubs.

For switching repos, use something like: micropy stubs repos and allow users to select an active repo via a checkbox.

Update Notifier favors release candidate over release

Describe the bug
When a new release comes out that has had one or more release candidates before it, micropy-cli notifies that an update is available with the latest release candidate instead the actual release.

Expected Behavior
micropy-cli should notify the user of a newer version of itself and display said version.

Current Behavior
micropy-cli will display the most recent release candidate for the new release instead of the new release.

Steps to Reproduce

Create release candidate (ex. v3.2.1.rc.1)
Create a new release (ex. v3.2.1
When notifying those on an older version, micropy-cli will display v3.2.1.rc.1 rather than v3.2.1

Possible Solution
When parsing the versions in micropy.utils.is_update_available, ignore any with "rc" in the name.

micropy init fails on Win10

I installed your awesome micropy-cli on Windows 10 following LeMaRiva’s guide and your readme.
Installation and stub creation went fine.

But ”micropy init” fails as seen below.
(I have different problems when I try to run the cmd in a folder on my NAS shared drive – but thats another story and not important right now)

VSCode is in my path and I can launch it from the cmd line.

VSCODE version info:
Version: 1.39.0 (user setup)
Commit: 9df03c6d6ce97c6645c5846f6dfa2a6a7d276515
Date: 2019-10-09T06:58:04.439Z
Electron: 4.2.10
Chrome: 69.0.3497.128
Node.js: 10.11.0
V8: 6.9.427.31-electron.0
OS: Windows_NT x64 10.0.17763

Python: 3.7.4,

VS Code extension:
Name: Python
Id: ms-python.python
Description: Linting, Debugging (multi-threaded, remote), Intellisense, code formatting, refactoring, unit tests, snippets, and more.
Version: 2019.10.41019
Publisher: Microsoft
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=ms-python.python

I hope you can help me out.

Thanks,

Lars Østergaard Knudsen
Copenhagen, Denmark

C:\Users\famil\Documents\Projects\MicroPython>micropy init hello1

MicroPy Creating New Project
? Choose any Templates to Generate [VSCode Settings for Autocompletion/Intellisense]
? Which stubs would you like to use? [esp32-micropython-1.11.0]

MicroPy Initiating hello1
MicroPy Stubs: esp32-micropython-1.11.0

MicroPy Rendering Templates
MicroPy Populating Stub info...
Traceback (most recent call last):
File "c:\program files\python37\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\program files\python37\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Program Files\Python37\Scripts\micropy.exe_main.py", line 9, in
File "c:\program files\python37\lib\site-packages\click\core.py", line 764, in call
return self.main(*args, **kwargs)
File "c:\program files\python37\lib\site-packages\click\core.py", line 717, in main
rv = self.invoke(ctx)
File "c:\program files\python37\lib\site-packages\click\core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\program files\python37\lib\site-packages\click\core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\program files\python37\lib\site-packages\click\core.py", line 555, in invoke
return callback(*args, **kwargs)
File "c:\program files\python37\lib\site-packages\micropy\cli.py", line 97, in init
proj_relative = project.create()
File "c:\program files\python37\lib\site-packages\micropy\project\project.py", line 364, in create
self.render_all()
File "c:\program files\python37\lib\site-packages\micropy\project\project.py", line 343, in render_all
self.provider.render_to(t, self.path, **self.context)
File "c:\program files\python37\lib\site-packages\micropy\project\template.py", line 256, in render_to
template.run_checks()
File "c:\program files\python37\lib\site-packages\micropy\project\template.py", line 60, in run_checks
results = [not ck() for ck in self.CHECKS]
File "c:\program files\python37\lib\site-packages\micropy\project\template.py", line 60, in
results = [not ck() for ck in self.CHECKS]
File "c:\program files\python37\lib\site-packages\micropy\project\checks.py", line 47, in vscode_ext_min_version
name, vers = next(iter_vscode_ext(name=ext), (ext, '0.0.0'))
File "c:\program files\python37\lib\site-packages\micropy\project\checks.py", line 24, in iter_vscode_ext
proc = subproc.run(_cmd.split(), text=True, capture_output=True)
File "c:\program files\python37\lib\subprocess.py", line 472, in run
with Popen(*popenargs, **kwargs) as process:
File "c:\program files\python37\lib\subprocess.py", line 775, in init
restore_signals, start_new_session)
File "c:\program files\python37\lib\subprocess.py", line 1178, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] Den angivne fil blev ikke fundet

Refactor Pyboard Stub Creation

At least:
Refactor with rshell, moving the functionality out of main.MicroPy .

Maybe later:
Remove rshell dep entirely, just use straight up pyboard.py

Tests:
Figure out some tests for it

Add Pyboard/Rshell Wrapper
Add Wrapper Tests
Refactor main.Micropy.create_stubs
Add tests

Micropy install should install packages

I'm opening this in the hopes of a fruitful conversation... my proposal may not be the best path.

Is your feature request related to a problem? Please describe.
micropy install doesn't install packages, it installs stubs.
This is counter intuitive to how "install" is used in the Python world. Issue #106 is an example of this, my guess you've had this conversation before with others.

Describe the solution you'd like
micropy should actually do the install and should install packages into src/lib/ as the default behavior.

Describe alternatives you've considered
One option would be to rename install to micropy stub-package. but it would be the most awesome and helpful if it actually did the install.

Additional context
There are a couple of approaches to look into:

Call micropython -m upip directly.
Merge micropip.py into micropy-cli

I believe option 1 is less fragile long-term because of the danger-warning Peter puts in the comments of micropip.py. I submitted a PR today to MP to add Unix Port installation instructions without doing a build. You can easily install micropython on Linux and MacOS. Windows is unclear. Checks would need to be added to verify micropython is installed and micropython would then need to fail gracefully.

#2 is cleaner and has less dependencies. It should be possible to fix micropip's logic when it is merged to account for Paul's concerns (which I don't yet understand completely).

I'm curious what you think Braden..

micropy install dependency issue

I'm new to micropython, so this may well be the fault of inexperience but here goes:

I'm using your great tool under Windows and VS Code.

I'm trying to run some code that depends on urlli.parse and urllib.parse depends on a module called collections.

I have run 'micropy install micropython-urllib.parse' and 'micropy install micropython-collections'

However, once uploaded to my pycom fipy I keep getting the error

File "/flash/lib/parse.py", line 32, in <module>
ImportError: no module named 'collections'

However:

>>> os.listdir('lib')
['micropython-collections', 'micropython-hashlib', 'base64.py', 'defaultdict.py', 'hmac.py', 'mqtt_async.py', 'parse.py', 'urllib.py', 'warnings.py']

Any idea where I'm going wrong?

Add more Docs

Update README and add more documentation.

Getting jsondecodeerror

When I try to install something in VScode , for example, picoweb, I get the following error:

$ micropy install picoweb

MicroPy  Loading Project
MicroPy  esp32-micropython-1.11.0 is already installed!

MicroPy  Fetching Requirements
Traceback (most recent call last):
  File "/anaconda3/bin/micropy", line 10, in <module>
    sys.exit(cli())
  File "/anaconda3/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/anaconda3/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/anaconda3/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/anaconda3/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/anaconda3/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/anaconda3/lib/python3.7/site-packages/micropy/cli.py", line 120, in install
    project = Project.resolve('.')
  File "/anaconda3/lib/python3.7/site-packages/micropy/project/project.py", line 388, in resolve
    proj.load(stub_manager=micropy.STUBS, verbose=verbose)
  File "/anaconda3/lib/python3.7/site-packages/micropy/project/project.py", line 195, in load
    self.load_packages()
  File "/anaconda3/lib/python3.7/site-packages/micropy/project/project.py", line 163, in load_packages
    meta = utils.get_package_meta(name, spec=spec)
  File "/anaconda3/lib/python3.7/site-packages/micropy/utils/helpers.py", line 259, in get_package_meta
    data = resp.json()
  File "/anaconda3/lib/python3.7/site-packages/requests/models.py", line 897, in json
    return complexjson.loads(self.text, **kwargs)
  File "/anaconda3/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/anaconda3/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/anaconda3/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 7 column 1 (char 6)

Get the error with any package I'm trying to install.

Improve pylint path syntax and add ./src/lib

Hi Braden,

I would like to suggest :

a change in syntax to the generated .pylintrc file to make it cleared to read and modify.
the addition of src/lib path as this is commonly used

current syntax (windows paths )

[MASTER]
# Loaded Stubs:  esp32-micropython-1.11.0 
init-hook='import sys;sys.path.insert(1, ".micropy\\BradenM-micropy-stubs-7a7d171\\frozen");sys.path.insert(1, ".micropy\\BradenM-micropy-stubs-452a204\\frozen");sys.path.insert(1, ".micropy\\BradenM-micropy-stubs-7a7d171\\stubs");sys.path.insert(1,"src\\lib");sys.path.insert(1,"src\\font")'

...

new syntax

[MASTER]
# Loaded Stubs:  esp32-micropython-1.11.0 
init-hook='import sys;sys.path[1:1] = ["src/lib", "src/fonts", ".micropy/BradenM-micropy-stubs-7a7d171/frozen", ".micropy/BradenM-micropy-stubs-452a204/frozen", ".micropy/BradenM-micropy-stubs-7a7d171/stubs",".micropy/thermo"]'
# Note:  when adding paths; also add these paths to: VSCode Preferences > Workspace > python.autoComplete.extraPaths

...

The new syntax has the paths in the right order , is shorter, is simpler to extend with additional paths.

For me this helps in organizing my modules. I think this should be useful to others as well.

/Jos

Handle Frozen Modules

Once remote stubs are complete and MicropyCli is using micropy-stubs as its primary stub source, frozen modules will be downloaded with the "stub packages." So, we can then handle adding frozen modules to better vscode intellisense.

Blocked by #7

Split UI and Logging Functionality

Refactoring logger.py and separate terminal UI from logging.

Add Recommended extensions to VS Code settings

Generate an extensions.json file with the required extensions for micropy if VS Code integration is used.

See Workspace Recommended Extensions

ms-python.python not recognized

When running MicroPy init, I get the following error:

MicroPy  VSCode Extension ms-python.python failed to satisfy requirements!
MicroPy  Min Required Version: 2019.9.34474
MicroPy  Current Version: 0.0.0
MicroPy  VSCode Integration will fail! See BradenM/micropy-cli#50 for details.

despite having 2020.2.64397 installed.

After running the init I get prompted to select a Python interpreter, which is possibly why it's not found?

OS: Mac OSx Catalina 10.15.3
Micropy Version: 3.3.0
Python Version: 3.7.3
VSCode Version: 1.43.0

TypeError when adding stubs to existing project

Describe the bug
When using micropy stubs add to add a stubs to an existing project, a TypeError is raised. This does not impact using micropy add to install stubs normally.

Expected Behavior
In an active project, micropy stubs add should install the stubs and add them to the active project.

Current Behavior
In an active project, the stubs install but a TypeError is raised when attempting to add them to the project.

Steps to Reproduce

Create New Project
cd to New Project
Execute micropy stubs add <some_stub_package>

Logs

Traceback:

Traceback (most recent call last):
  File "/home/bradenmars/.local/bin/micropy", line 10, in <module>
    sys.exit(cli())
  File "/home/bradenmars/.local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/bradenmars/.local/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/bradenmars/.local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/bradenmars/.local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/bradenmars/.local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/bradenmars/.local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/bradenmars/.local/lib/python3.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/bradenmars/.local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/bradenmars/.local/lib/python3.7/site-packages/micropy/cli.py", line 216, in add
    proj.add_stub(stub)
  File "/home/bradenmars/.local/lib/python3.7/site-packages/micropy/project/modules/modules.py", line 147, in __call__
    return getattr(*proxy)(*args, **kwargs)
  File "/home/bradenmars/.local/lib/python3.7/site-packages/micropy/project/modules/modules.py", line 96, in wrapper
    return func(*args, **kwargs)
  File "/home/bradenmars/.local/lib/python3.7/site-packages/micropy/project/modules/stubs.py", line 150, in add_stub
    self.context.extend('stubs', [stub])
  File "/home/bradenmars/.local/lib/python3.7/site-packages/micropy/config/config.py", line 177, in extend
    dpath.merge(to_update, value, flags=dpath.MERGE_ADDITIVE)
  File "/home/bradenmars/.pyenv/versions/3.7.3/lib/python3.7/site-packages/dpath/util.py", line 213, in merge
    dst += [None] * (dsti - (len(dst) - 1))
TypeError: unsupported operand type(s) for +=: 'set' and 'list'

Stub Loading is Slow and Repetitive

Currently, loading stubs requires validating the same stubs multiple times. Since the stub JSON file is read from on each validation, this is a very slow and repetitive process. When you have ~5 or more stub packages, executing micropy tends to have a very noticeable delay.

A much faster solution would be to only validate when a stub is added (not just loaded from the stubs folder) then create and save an md5 hash of the stub info file along with what type of stub it is. We can then "validate" and determine existing stubs by reading from this one file, rather than re-validating them all.

Micropy Project Folder

Instead of using $HOME/.micropy to store any stubs added to micropy, create a VCS ignored .micropy folder at the root of a project. Similiar to a node_modules folder, except for storing stubs/frozen modules used in the project.

Could possibly use the current $HOME/.micropy folder to store all added stubs, but just add symlinks in the PROJECT/.micropy folder.

This would be a much better and more flexible solution than #17.

Blocked by #9
Blocked by #7

MS-Python 2019.9.34474 breaks VSCode functionality

The Problem

Microsoft recently released v2019.9.34911 of the official Python extension for VSCode.

microsoft/vscode-python#7024 was included, which deprecated DonJayamanne's workaround allowing the ${workspaceRoot} folder to be used in .vscode/settings.json.

Since micropy-cli used this variable to point vscode to a project's local .micropy/ folder, vscode integration silently fails when its passed along to the Python Language Server:

Python Language Server Log

[Info  - 8:40:35 PM] Microsoft Python Language Server version 0.3.76.0
  -- truncated --
[Info  - 8:40:35 PM] User search paths:
[Info  - 8:40:35 PM]     .../ProjectTest/${workspaceRoot}/.micropy/pybricks/frozen
[Info  - 8:40:35 PM]     .../ProjectTest/${workspaceRoot}/.micropy/ev3-pybricks-v1.0.0/frozen
[Info  - 8:40:35 PM]    .../ProjectTest/${workspaceRoot}/.micropy/ev3-pybricks-v1.0.0/stubs

The Solution

Check your micropy-cli version by executing micropy --version

v2.1.1 AND UP

A hotfix for this issue will be available in v2.1.1
Will try and release it as soon as I can.

Hotfix released: Please upgrade via pip install --upgrade micropy-cli

v0.0.0 TO v2.1.0 (INCLUSIVE):

OPTION 1. Update micropy-cli via pip install --upgrade micropy-cli

OPTION 2. Install an older version of MS-Python from their Github releases page. See how to install a .vsix file here.

OPTION 3. Until a hotfix is released, you many temporarily resolve this problem by manually updating your .vscode/settings.json file by removing all instances of "${workspaceRoot}/"

Example:

Original:

{
    "python.linting.enabled": true,
    "python.autoComplete.extraPaths": [
        "${workspaceRoot}/.micropy/pybricks/frozen",
        "${workspaceRoot}/.micropy/ev3-pybricks-v1.0.0/stubs"
    ],
    "python.autoComplete.typeshedPaths": [
        "${workspaceRoot}/.micropy/pybricks/frozen",
        "${workspaceRoot}/.micropy/ev3-pybricks-v1.0.0/stubs"
    ],
    "python.analysis.typeshedPaths": [
        "${workspaceRoot}/.micropy/pybricks/frozen",
        "${workspaceRoot}/.micropy/ev3-pybricks-v1.0.0/stubs"
    ],
    "python.linting.pylintEnabled": true
}

After:

{
    "python.linting.enabled": true,
    "python.autoComplete.extraPaths": [
        ".micropy/pybricks/frozen",
        ".micropy/ev3-pybricks-v1.0.0/stubs"
    ],
    "python.autoComplete.typeshedPaths": [
        ".micropy/pybricks/frozen",
        ".micropy/ev3-pybricks-v1.0.0/stubs"
    ],
    "python.analysis.typeshedPaths": [
        ".micropy/pybricks/frozen",
        ".micropy/ev3-pybricks-v1.0.0/stubs"
    ],
    "python.linting.pylintEnabled": true
}

Not work on Pyboard (and clones)

micropy-cli version 2.1.0 not work on Pyboard (STM32F405RGT6 MCU) and it's clones.
stubs generation end with errors. On ESP32 work without problems.

micropy stubs create COM8
...
MicroPy  Pyboard  Stub module: writer
MicroPy  Pyboard  Stub module: ymodem
MicroPy  Pyboard  Stub module: zlib
MicroPy  ✔ Done!
MicroPy  Copying stubs...
Source directory /pyboard/stubs does not exist.
Traceback (most recent call last):
  File "d:\applications\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "d:\applications\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\Applications\Python37\Scripts\micropy.EXE\__main__.py", line 9, in <module>
  File "d:\applications\python37\lib\site-packages\click\core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "d:\applications\python37\lib\site-packages\click\core.py", line 717, in main
    rv = self.invoke(ctx)
  File "d:\applications\python37\lib\site-packages\click\core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "d:\applications\python37\lib\site-packages\click\core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "d:\applications\python37\lib\site-packages\click\core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "d:\applications\python37\lib\site-packages\click\core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "d:\applications\python37\lib\site-packages\micropy\cli.py", line 224, in create
    return mp.create_stubs(port, verbose=verbose)
  File "d:\applications\python37\lib\site-packages\micropy\main.py", line 83, in create_stubs
    stub_path = next(out_dir.iterdir())
StopIteration

After physically device disconnect-connect there is stub files visible on filesystem.
Main difference of this boards is internal fs exposed as USB mass storage device. Disabling USB MSD not solve this problem.

Creatu stub fails on MemoryError

Describe the bug
Tried to create stub for Pycopy v3.0.6 on esp8266

Expected Behavior
It will create stub.

Current Behavior

> micropy stubs create /dev/cu.SLAB_USBtoUART

MicroPy  Connecting to Pyboard @ /dev/cu.SLAB_USBtoUART
MicroPy  ✔ Connected!
MicroPy  Executing stubber on pyboard...
MicroPy  Failed to execute script: ('exception', b'', b'MemoryError: memory allocation failed, allocating %u bytes\r\n')

Steps to Reproduce

Upload Pycopy firmware
micropy stubs create

Logs

[2020-03-29 04:47:12] DEBUG: hookproxy(add_from_file): Method added to proxy: (<function PackagesModule.add_from_file at 0x109c14cb0>, '_hook__add_from_file__path_None__dev_False')
[2020-03-29 04:47:12] DEBUG: hookproxy(add_package): Method added to proxy: (<function PackagesModule.add_package at 0x109c14e60>, '_hook__add_package__dev_False')
[2020-03-29 04:47:12] DEBUG: hookproxy(add_package): Method added to proxy: (<function DevPackagesModule.add_package at 0x109c1f320>, '_hook__add_package__dev_True')
[2020-03-29 04:47:12] DEBUG: hookproxy(add_from_file): Method added to proxy: (<function DevPackagesModule.add_from_file at 0x109c1f440>, '_hook__add_from_file__path_None__dev_True')
[2020-03-29 04:47:12] DEBUG: hookproxy(stubs): Method added to proxy: (<function StubsModule.stubs at 0x109c47320>, '_hook__stubs__')
[2020-03-29 04:47:12] DEBUG: hookproxy(add_stub): Method added to proxy: (<function StubsModule.add_stub at 0x109c49950>, '_hook__add_stub__')
[2020-03-29 04:47:12] DEBUG: micropy: MicroPy Loaded
[2020-03-29 04:47:12] INFO: micropy: Connecting to Pyboard @ /dev/cu.SLAB_USBtoUART
[2020-03-29 04:47:12] DEBUG: pyboard: connecting to pydevice @ /dev/cu.SLAB_USBtoUART
[2020-03-29 04:47:13] DEBUG: pyboard: connected!
[2020-03-29 04:47:13] INFO: micropy: Connected!
[2020-03-29 04:47:13] INFO: micropy: Executing stubber on pyboard...
[2020-03-29 04:47:15] DEBUG: pyboard: Failed to run script on pyboard: ('exception', b'', b'MemoryError: memory allocation failed, allocating %u bytes\r\n')
[2020-03-29 04:47:15] ERROR: micropy: Failed to execute script: ('exception', b'', b'MemoryError: memory allocation failed, allocating %u bytes\r\n')

Context (Environment)

OS: macOS 10.15.3 (19D76)
Micropy Version: 3.3.0
Python Version: 2.7.16

<SYMLINKD> is missing when make a git Clone

Hi
When I get an GIT Clone of my project on new PC, the for -esp-micropython-1.11.0 and micropython is not created.

Can not install micropython-wiegand

Describe the bug

Tried to install specific package from pypi, but it failed.

Different packages worked.

Expected Behavior

It will instlall the package.
Current Behavior

It failed on error.
Steps to Reproduce

micropy install micropython-wiegand

Also tried to install from URL to zip or raw file.

Logs

 $ pycopy > micropy install micropython-wiegand       
MicroPy  Loading Project
MicroPy  ✔ Ready!

MicroPy  Installing Packages
Traceback (most recent call last):
  File "/usr/local/bin/micropy", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/cli.py", line 171, in install
    project.add_package(pkg, dev=dev)
  File "/usr/local/lib/python3.7/site-packages/micropy/project/modules/modules.py", line 147, in __call__
    return getattr(*proxy)(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/project/modules/modules.py", line 96, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/project/modules/packages.py", line 142, in add_package
    source = create_dependency_source(package, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/__init__.py", line 43, in create_dependency_source
    return PackageDependencySource(pkg, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/source_package.py", line 39, in __init__
    self.repo_url
  File "/usr/local/lib/python3.7/site-packages/micropy/utils/helpers.py", line 287, in get_package_meta
    tar_meta = next((i for i in spec_data if ".tar.gz" in Path(i['url']).name))
StopIteration

 $ pycopy > micropy install https://files.pythonhosted.org/packages/b8/e0/874eed0203d5051fc255ea16fbf43d4cc08ddbdd66de1c46bd4ce9a02c6b/micropython-wiegand-1.1.0.zip

MicroPy  Loading Project
MicroPy  ✔ Ready!

MicroPy  Installing Packages
Traceback (most recent call last):
  File "/usr/local/bin/micropy", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/cli.py", line 171, in install
    project.add_package(pkg, dev=dev)
  File "/usr/local/lib/python3.7/site-packages/micropy/project/modules/modules.py", line 147, in __call__
    return getattr(*proxy)(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/project/modules/modules.py", line 96, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/project/modules/packages.py", line 142, in add_package
    source = create_dependency_source(package, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/__init__.py", line 43, in create_dependency_source
    return PackageDependencySource(pkg, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/source_package.py", line 30, in __init__
    super().__init__(package)
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/source.py", line 28, in __init__
    self.log: ServiceLog = Log.add_logger(repr(self))
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/source.py", line 79, in __repr__
    return f"<{self.__class__.__name__} {self.package}>"
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/package.py", line 80, in __str__
    return self.full_name
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/package.py", line 42, in full_name
    return self.name
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/package.py", line 29, in name
    return canonicalize_name(self._name)
  File "/usr/local/lib/python3.7/site-packages/packaging/utils.py", line 16, in canonicalize_name
    return _canonicalize_regex.sub("-", name).lower()
TypeError: expected string or bytes-like object

 $ pycopy > micropy install https://github.com/pjz/micropython-wiegand/archive/1.1.0.tar.gz

MicroPy  Loading Project
MicroPy  ✔ Ready!

MicroPy  Installing Packages
Traceback (most recent call last):
  File "/usr/local/bin/micropy", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/cli.py", line 171, in install
    project.add_package(pkg, dev=dev)
  File "/usr/local/lib/python3.7/site-packages/micropy/project/modules/modules.py", line 147, in __call__
    return getattr(*proxy)(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/project/modules/modules.py", line 96, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/project/modules/packages.py", line 142, in add_package
    source = create_dependency_source(package, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/__init__.py", line 43, in create_dependency_source
    return PackageDependencySource(pkg, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/source_package.py", line 30, in __init__
    super().__init__(package)
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/source.py", line 28, in __init__
    self.log: ServiceLog = Log.add_logger(repr(self))
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/source.py", line 79, in __repr__
    return f"<{self.__class__.__name__} {self.package}>"
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/package.py", line 80, in __str__
    return self.full_name
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/package.py", line 42, in full_name
    return self.name
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/package.py", line 29, in name
    return canonicalize_name(self._name)
  File "/usr/local/lib/python3.7/site-packages/packaging/utils.py", line 16, in canonicalize_name
    return _canonicalize_regex.sub("-", name).lower()
TypeError: expected string or bytes-like object

 $ pycopy > micropy install https://raw.githubusercontent.com/pjz/micropython-wiegand/master/wiegand.py
MicroPy  Loading Project
MicroPy  ✔ Ready!

MicroPy  Installing Packages
Traceback (most recent call last):
  File "/usr/local/bin/micropy", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/cli.py", line 171, in install
    project.add_package(pkg, dev=dev)
  File "/usr/local/lib/python3.7/site-packages/micropy/project/modules/modules.py", line 147, in __call__
    return getattr(*proxy)(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/project/modules/modules.py", line 96, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/project/modules/packages.py", line 142, in add_package
    source = create_dependency_source(package, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/__init__.py", line 43, in create_dependency_source
    return PackageDependencySource(pkg, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/source_package.py", line 30, in __init__
    super().__init__(package)
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/source.py", line 28, in __init__
    self.log: ServiceLog = Log.add_logger(repr(self))
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/source.py", line 79, in __repr__
    return f"<{self.__class__.__name__} {self.package}>"
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/package.py", line 80, in __str__
    return self.full_name
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/package.py", line 42, in full_name
    return self.name
  File "/usr/local/lib/python3.7/site-packages/micropy/packages/package.py", line 29, in name
    return canonicalize_name(self._name)
  File "/usr/local/lib/python3.7/site-packages/packaging/utils.py", line 16, in canonicalize_name
    return _canonicalize_regex.sub("-", name).lower()
TypeError: expected string or bytes-like object

Context (Environment)

macOS

micropy, version 3.3.0

Python 2.7.16 default in the system.
Also installed Python 3.7.7, but micropy probably does not use it.

PyCharm IDE Support

Should be able to support PyCharm by handling a .idea/micropy.iml file included in .idea/modules.xml that sources stubs from a projects local .micropy folder.

Option to upload dependencies via /lib folder

Add configuration option to micropy.json to enable uploading project dependencies via a /lib folder.

See this comment for more.

Add Tests for cli module

VSCode use micropy-cli

Hi, I want to use micropy-cli as vscode Intellisense.In my configuration process, I have two questions:

when I init project,micropy init new raise a exception.

requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

when I finish init project,I find vscode can`t autocompletion.bec my project`s .micropy is void.when i read the doc,i find my ~/.micropy is different of the doc show.

I very much hope to get yours help.thanks!

Nonexisting packages throw error

Commit: ae3b811

Steps to reproduce:

micropy init testproject
micropy install nonexistingpackage

This gives some errors, leading back to utils/helpers.py in function:
def get_package_meta(name, spec=None):

279 url = f"https://pypi.org/pypi/{name}/json"
280 resp = requests.get(url)

I don't know what the right moment would be to check if the package exists.

Log:

[2019-12-10 12:21:51] INFO: micropy: Installing Packages
[2019-12-10 12:21:51] DEBUG: testproject: Hook Resolved: add_package -> HookProxy(add_package)
[2019-12-10 12:21:51] DEBUG: hookproxy(add_package): add_package proxied to [_hook__add_package__dev_False@<micropy.project.modules.packages.PackagesModule object at 0x00000283D07ABF60>]
[2019-12-10 12:21:51] DEBUG: testproject: adding new dependency: nonexistingpackage
[2019-12-10 12:21:51] DEBUG: <packagedependencysource nonexistingpackage>: Starting new HTTPS connection (1): pypi.org:443
[2019-12-10 12:21:51] DEBUG: <packagedependencysource nonexistingpackage>: https://pypi.org:443 "GET /pypi/nonexistingpackage/json HTTP/1.1" 301 227
[2019-12-10 12:21:52] DEBUG: <packagedependencysource nonexistingpackage>: https://pypi.org:443 "GET /pypi/nonexistingpackage/json/ HTTP/1.1" 404 1839

Traceback:

Traceback (most recent call last):
  File "c:\users\user\appdata\local\programs\python\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\user\appdata\local\programs\python\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37\Scripts\micropy.exe\__main__.py", line 7, in <module>
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 717, in main
    rv = self.invoke(ctx)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\click\decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\micropy\cli.py", line 168, in install
    project.add_package(pkg, dev=dev)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\micropy\project\modules\modules.py", line 147, in __call__
    return getattr(*proxy)(*args, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\micropy\project\modules\modules.py", line 96, in wrapper
    return func(*args, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\micropy\project\modules\packages.py", line 136, in add_package
    source = create_dependency_source(package, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\micropy\packages\__init__.py", line 43, in create_dependency_source
    return PackageDependencySource(pkg, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\micropy\packages\source_package.py", line 30, in __init__
    self._meta: dict = utils.get_package_meta(str(self.package), self.package.pretty_specs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\micropy\utils\helpers.py", line 283, in get_package_meta
    data = resp.json()
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\requests\models.py", line 897, in json
    return complexjson.loads(self.text, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\json\__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "c:\users\user\appdata\local\programs\python\python37\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "c:\users\user\appdata\local\programs\python\python37\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 7 column 1 (char 6)

Project on exFAT partition on Ubuntu > OSError: [Errno 38] Function not implemented

Describe the bug

lemariva@:~/Documents/work/lemariva.com/Github/MicroPy$ micropy init

MicroPy  Creating New Project
? Project Name MicroPy                                                                                                                              
? Choose any Templates to Generate  [VSCode Settings for Autocompletion/Intellisense]                                                               
? Which stubs would you like to use?  [esp32-micropython-1.12.0]                                                                                    

MicroPy  Initiating MicroPy
MicroPy  Stubs: esp32-micropython-1.12.0
MicroPy  Failed to Create Stub Links!
MicroPy  OSError: [Errno 38] Function not implemented: '/home/lemariva/.micropy/stubs/BradenM-micropy-stubs-27ee48a' -> '/home/lemariva/Documents/work/lemariva.com/Github/MicroPy/.micropy/BradenM-micropy-stubs-27ee48a'
lemariva@l:~/Documents/work/lemariva.com/Github/MicroPy$

The problem appears on Ubuntu (20.04 LTS).
An exFAT partition is mounted on '/home/lemariva/Documents/work/' (encrypted and shared with Windows 10 - that's why exFAT). The exFAT file system does not support soft/hard links. The line:

source.symlink_to(target, target_is_directory=True)

generates the exception.

Possible Solution
If the error occurs (errno 38), copy the folders instead of making a link.

Logs

[2020-05-05 00:15:09] INFO: stubs: esp32-micropython-1.12.0 is already installed!
[2020-05-05 00:15:09] DEBUG: micropy.config.config: added config value [stubs] -> {'esp32-micropython-1.12.0': '1.3.2'}
[2020-05-05 00:15:09] ERROR: micropy: Failed to Create Stub Links!
[2020-05-05 00:15:09] ERROR: micropy: OSError: [Errno 38] Function not implemented: '/home/lemariva/.micropy/stubs/BradenM-micropy-stubs-27ee48a' -> '/home/lemariva/Documents/work/lemariva.com/Github/MicroPy/.micropy/BradenM-micropy-stubs-27ee48a'
Traceback (most recent call last):
  File "/home/lemariva/.local/lib/python3.8/site-packages/micropy/project/modules/stubs.py", line 90, in _resolve_subresource
    resource = set(
  File "/home/lemariva/.local/lib/python3.8/site-packages/micropy/stubs/stubs.py", line 381, in resolve_subresource
    fware = FirmwareStub.resolve_link(fware, link)
  File "/home/lemariva/.local/lib/python3.8/site-packages/micropy/stubs/stubs.py", line 435, in resolve_link
    utils.create_dir_link(link_path, stub.path)
  File "/home/lemariva/.local/lib/python3.8/site-packages/micropy/utils/helpers.py", line 334, in create_dir_link
    raise e
  File "/home/lemariva/.local/lib/python3.8/site-packages/micropy/utils/helpers.py", line 329, in create_dir_link
    source.symlink_to(target, target_is_directory=True)
  File "/usr/lib/python3.8/pathlib.py", line 1374, in symlink_to
    self._accessor.symlink(target, self, target_is_directory)
  File "/usr/lib/python3.8/pathlib.py", line 442, in symlink
    return os.symlink(a, b)
OSError: [Errno 38] Function not implemented: '/home/lemariva/.micropy/stubs/BradenM-micropy-stubs-27ee48a' -> '/home/lemariva/Documents/work/lemariva.com/Github/MicroPy/.micropy/BradenM-micropy-stubs-27ee48a'

Context (Environment)

OS: Ubuntu 20.04 LTS
Micropy Version: micropy, version 3.3.0
Python Version: Python 3.8.2
VSCode Version: 1.45.0-insider

micropy overwrites existing .vscode/settings.json

Describe the bug
micropy init will overwrite .vscode/settings.json in a workspace with existing settings

Expected Behavior
Micropy should add itself to the workspace settings, only overwriting specific settings entries.

Current Behavior
micropy over writes the existing .vscode/settings.json loosing user preferences.

Steps to Reproduce

create a new folder.
launch code from that folder code .
From code make a change that creates a workspace settings file. For example use Peacock to change the IDE color. At this point I have the following settings.json:

{
    "workbench.colorCustomizations": {
        "activityBar.activeBackground": "#65c89b",
        "activityBar.activeBorder": "#945bc4",
        "activityBar.background": "#65c89b",
        "activityBar.foreground": "#15202b",
        "activityBar.inactiveForeground": "#15202b99",
        "activityBarBadge.background": "#945bc4",
        "activityBarBadge.foreground": "#e7e7e7",
        "statusBar.background": "#42b883",
        "statusBar.border": "#42b883",
        "statusBar.foreground": "#15202b",
        "statusBarItem.hoverBackground": "#359268",
        "titleBar.activeBackground": "#42b883",
        "titleBar.activeForeground": "#15202b",
        "titleBar.border": "#42b883",
        "titleBar.inactiveBackground": "#42b88399",
        "titleBar.inactiveForeground": "#15202b99"
    },
    "peacock.color": "#42b883"
}

Back to the CLI, run micropy init . and include the VS Code settings in your choices. I now have the follwing settings.json file:

{
    "python.linting.enabled": true,
    "python.jediEnabled": false,

    // Loaded Stubs:  esp32-micropython-1.12.0 
    "python.autoComplete.extraPaths": [".micropy/BradenM-micropy-stubs-9569403/frozen", ".micropy/BradenM-micropy-stubs-27ee48a/frozen", ".micropy/BradenM-micropy-stubs-9569403/stubs", ".micropy/bug-repo"],
    "python.autoComplete.typeshedPaths":  [".micropy/BradenM-micropy-stubs-9569403/frozen", ".micropy/BradenM-micropy-stubs-27ee48a/frozen", ".micropy/BradenM-micropy-stubs-9569403/stubs", ".micropy/bug-repo"],
    "python.analysis.typeshedPaths":  [".micropy/BradenM-micropy-stubs-9569403/frozen", ".micropy/BradenM-micropy-stubs-27ee48a/frozen", ".micropy/BradenM-micropy-stubs-9569403/stubs", ".micropy/bug-repo"],

    "python.linting.pylintEnabled": true
}

Possible Solution
NOTE: If you have a main.py already in the workspace it will not be overwritten for that init option.

Logs
N\A

Context (Environment)

OS: MacOS
Micropy Version: micropy, version 3.3.0
Python Version: 3.8.2
VSCode Version: 1.45.1 - 5763d909d5f12fe19f215cbfdd29a91c0fa9208a - x64

Can't resolve import?

Thank you for this project. I am a novice when it comes to micropython and I found your tool to help me get going quickly. I am trying to use the micropython bluetooth library. VS Code is giving me an error however saying that it cannot be found. I tried running micropy install bluetooth as well but it said it could not be found. What exactly do I need to do to use the bluetooth library. Please see the screenshot below. Thanks again.

Set PYTHONPATH in VS Code Settings

If project uses VS Code integration, render terminal.integrated.env.{PLATFORM} setting in .vscode/settings.json with PYTHONPATHcontaining paths to dependencies.

This will allow users to import these dependencies from the VS Code integrated terminal just like in their code.

See Use of the PYTHONPATH variable for more details.

failed to execute micropy-cli on windows 10

Hello, just try to execute the micropy cli on windows 10 with the command "micropy stubs search esp32" and i've got the following stack trace

Traceback (most recent call last):
  File "c:\users\jtutt\appdata\local\programs\python\python36-32\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\jtutt\appdata\local\programs\python\python36-32\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\jtutt\AppData\Local\Programs\Python\Python36-32\Scripts\micropy.exe\__main__.py", line 5, in <module>
  File "c:\users\jtutt\appdata\local\programs\python\python36-32\lib\site-packages\micropy\cli.py", line 13, in <module>
    from micropy.main import MicroPy
  File "c:\users\jtutt\appdata\local\programs\python\python36-32\lib\site-packages\micropy\main.py", line 9, in <module>
    from micropy import data, utils
  File "c:\users\jtutt\appdata\local\programs\python\python36-32\lib\site-packages\micropy\utils\__init__.py", line 12, in <module>
    from .pybwrapper import PyboardWrapper
  File "c:\users\jtutt\appdata\local\programs\python\python36-32\lib\site-packages\micropy\utils\pybwrapper.py", line 14, in <module>
    import rshell.main as rsh
  File "c:\users\jtutt\appdata\local\programs\python\python36-32\lib\site-packages\rshell\main.py", line 67, in <module>
    if 'libedit' in readline.__doc__:
TypeError: argument of type 'NoneType' is not iterable

Did you have any idea what is going wrong?

best regards

jtutt

Micropy Project File

Add a micropy.json file to each project to contain project info, stubs, modules, etc. Will be used similarly to a package.json.

Something like:

{
   "name":"Micropython Project",
   "firmware":{
      "micropython":"1.11"
   },
   "devices":[
      "esp8266",
      "esp32"
   ],
   "dependencies":{
      "blynklib":"0.2.4"
   }
}

Where firmware and devices will be used to install/use the project base stubs, and dependencies will be used to create (or be created from) requirements.txt and will automatically stub the module, adding it to the project.

This way we can make micropython projects more git friendly, as we could now do something like micropy init in the existing project folder (depends on #10) and have a workspace matching the original authors.

Blocked by #7
Blocked by #10

BUG: TypeError: Unknown color 'bright_green'

$ micropy --version
micropy, version 2.0.1

$ micropy init
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/click/termui.py", line 372, in style
    bits.append('\033[%dm' % (_ansi_colors.index(fg) + 30))
ValueError: tuple.index(x): x not in tuple

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/micropy", line 10, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/micropy/cli.py", line 63, in init
    mp = MicroPy()
  File "/usr/local/lib/python3.7/dist-packages/micropy/main.py", line 26, in __init__
    self.setup()
  File "/usr/local/lib/python3.7/dist-packages/micropy/main.py", line 30, in setup
    self.log.debug("\n---- MicropyCLI Session ----")
  File "/usr/local/lib/python3.7/dist-packages/micropy/logger.py", line 274, in debug
    return self.debug(msg, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/micropy/logger.py", line 275, in debug
    self.echo(msg, log="debug")
  File "/usr/local/lib/python3.7/dist-packages/micropy/logger.py", line 164, in echo
    service_title = self.get_service(fg=title_color, bold=title_bold)
  File "/usr/local/lib/python3.7/dist-packages/micropy/logger.py", line 135, in get_service
    f"{self.service_name}", fg=color, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/click/termui.py", line 374, in style
    raise TypeError('Unknown color %r' % fg)
TypeError: Unknown color 'bright_green'

$ sudo -H pip3.7 install -U micropy-cli
Requirement already up-to-date: micropy-cli in /usr/local/lib/python3.7/dist-packages (2.0.1)
Requirement already satisfied, skipping upgrade: click in /usr/local/lib/python3.7/dist-packages (from micropy-cli) (6.7)
Requirement already satisfied, skipping upgrade: jinja2 in /usr/local/lib/python3.7/dist-packages (from micropy-cli) (2.10)
Requirement already satisfied, skipping upgrade: requests in /usr/local/lib/python3.7/dist-packages (from micropy-cli) (2.22.0)
Requirement already satisfied, skipping upgrade: packaging in /usr/local/lib/python3.7/dist-packages (from micropy-cli) (19.1)
Requirement already satisfied, skipping upgrade: rshell in /usr/local/lib/python3.7/dist-packages (from micropy-cli) (0.0.26)
Requirement already satisfied, skipping upgrade: requirements-parser in /usr/local/lib/python3.7/dist-packages (from micropy-cli) (0.2.0)
Requirement already satisfied, skipping upgrade: jsonschema in /usr/local/lib/python3.7/dist-packages (from micropy-cli) (3.0.2)
Requirement already satisfied, skipping upgrade: tqdm in /usr/local/lib/python3.7/dist-packages (from micropy-cli) (4.24.0)
Requirement already satisfied, skipping upgrade: questionary in /usr/local/lib/python3.7/dist-packages (from micropy-cli) (1.2.1)
Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /usr/lib/python3/dist-packages (from jinja2->micropy-cli) (1.0)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests->micropy-cli) (2018.1.18)
Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /usr/lib/python3/dist-packages (from requests->micropy-cli) (3.0.4)
Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /usr/lib/python3/dist-packages (from requests->micropy-cli) (2.6)
Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/lib/python3/dist-packages (from requests->micropy-cli) (1.22)
Requirement already satisfied, skipping upgrade: attrs in /usr/local/lib/python3.7/dist-packages (from packaging->micropy-cli) (19.1.0)
Requirement already satisfied, skipping upgrade: six in /usr/local/lib/python3.7/dist-packages (from packaging->micropy-cli) (1.11.0)
Requirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /usr/local/lib/python3.7/dist-packages/pyparsing-2.2.0-py3.7.egg (from packaging->micropy-cli) (2.2.0)
Requirement already satisfied, skipping upgrade: pyudev>=0.16 in /usr/local/lib/python3.7/dist-packages (from rshell->micropy-cli) (0.21.0)
Requirement already satisfied, skipping upgrade: pyserial in /usr/local/lib/python3.7/dist-packages (from rshell->micropy-cli) (3.4)
Requirement already satisfied, skipping upgrade: setuptools in /usr/local/lib/python3.7/dist-packages (from jsonschema->micropy-cli) (40.0.0)
Requirement already satisfied, skipping upgrade: pyrsistent>=0.14.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema->micropy-cli) (0.15.4)
Requirement already satisfied, skipping upgrade: prompt-toolkit~=2.0 in /usr/local/lib/python3.7/dist-packages (from questionary->micropy-cli) (2.0.9)
Requirement already satisfied, skipping upgrade: wcwidth in /usr/local/lib/python3.7/dist-packages/wcwidth-0.1.7-py3.7.egg (from prompt-toolkit~=2.0->questionary->micropy-cli) (0.1.7)

Stub Manager

Seperate stub handling from Micropy Class to its own StubManager class.

Will make it much easier to add future things such as remote stub sources

micropy install error for local packages

Commit: ae3b811

Steps to reproduce:
micropy init testproject
micropy install -p ./src
micropy install

Error is different from issue #85 , but it does try to find the local package on pypi for some reason.

Log file:

[2019-12-10 12:33:48] DEBUG: hookproxy(add_package): add_package proxied to [_hook__add_package__dev_False@<micropy.project.modules.packages.PackagesModule object at 0x00000192D4D0C6A0>]
[2019-12-10 12:33:48] DEBUG: testproejct2: adding new dependency: src
[2019-12-10 12:33:48] DEBUG: <packagedependencysource src>: Starting new HTTPS connection (1): pypi.org:443
[2019-12-10 12:33:49] DEBUG: <packagedependencysource src>: https://pypi.org:443 "GET /pypi/src/json HTTP/1.1" 200 2342

Traceback:

Traceback (most recent call last):
  File "c:\users\user\appdata\local\programs\python\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\user\appdata\local\programs\python\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37\Scripts\micropy.exe\__main__.py", line 7, in <module>
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 717, in main
    rv = self.invoke(ctx)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\click\decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\micropy\cli.py", line 168, in install
    project.add_package(pkg, dev=dev)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\micropy\project\modules\modules.py", line 147, in __call__
    return getattr(*proxy)(*args, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\micropy\project\modules\modules.py", line 96, in wrapper
    return func(*args, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\micropy\project\modules\packages.py", line 136, in add_package
    source = create_dependency_source(package, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\micropy\packages\__init__.py", line 43, in create_dependency_source
    return PackageDependencySource(pkg, **kwargs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\micropy\packages\source_package.py", line 30, in __init__
    self._meta: dict = utils.get_package_meta(str(self.package), self.package.pretty_specs)
  File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\micropy\utils\helpers.py", line 297, in get_package_meta
    tar_meta = next((i for i in spec_data if ".tar.gz" in Path(i['url']).name))
StopIteration

Fix Failing Windows Tests

About 10 tests are currently failing on Windows.

UnicodeDecodeError on micropy init in Windows

I have install micropy-cli 1.1.2 on my Windows 10 (x64) where also python 3.7.3 (x64) is installed.
"Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information."

When I perform the following command "micropy-cli init test", I get the following error. I have also tried the same error on another PC.

Imicropy init test
Traceback (most recent call last):
File "c:\users\flemm\appdata\local\programs\python\python37\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\Flemm\AppData\Local\Programs\Python\Python37\Scripts\micropy.exe_main.py", line 9, in
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 764, in call
return self.main(*args, **kwargs)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 717, in main
rv = self.invoke(ctx)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 555, in invoke
return callback(*args, **kwargs)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\micropy\cli.py", line 54, in init
mp = MicroPy()
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\micropy\main.py", line 24, in init
self.setup()
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\micropy\main.py", line 36, in setup
self.log.debug(f"Creating .micropy directory @ {self.FILES}")
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\micropy\logger.py", line 274, in debug
return self.debug(msg, **kwargs)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\micropy\logger.py", line 275, in debug
self.echo(msg, log="debug")
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\micropy\logger.py", line 165, in echo
message, clean = self.parse_msg(msg, accent_color)
File "c:\users\flemm\appdata\local\programs\python\python37\lib\site-packages\micropy\logger.py", line 111, in parse_msg
clean = clean.encode('ascii', 'ignore').decode('unicode_escape')
UnicodeDecodeError: 'unicodeescape' codec can't decode bytes in position 32-33: truncated \UXXXXXXXX escape

bradenm / micropy-cli Goto Github PK

micropy-cli's People

Contributors

Stargazers

Watchers

Forkers

micropy-cli's Issues

OS

Micropy Version:

Python Version

VSCode Version

The Problem

The Solution

v2.1.1 AND UP

v0.0.0 TO v2.1.0 (INCLUSIVE):

Recommend Projects

Recommend Topics

Recommend Org