dropbox / stone Goto Github PK
View Code? Open in Web Editor NEWThe Official API Spec Language for Dropbox API V2
Home Page: https://www.dropbox.com/developers
License: MIT License
The Official API Spec Language for Dropbox API V2
Home Page: https://www.dropbox.com/developers
License: MIT License
I'm getting the following when running ./mypy-run.sh
as of 2e04e96:
stone/target/obj_c_types.py: note: In class "ObjCTypesGenerator":
stone/target/obj_c_types.py:99: error: Need type annotation for variable
stone/target/obj_c_client.py: note: In class "ObjCGenerator":
stone/target/obj_c_client.py:101: error: Need type annotation for variable
These didn't appear in 8fbb9e6.
Describe the bug
dropbox/dropbox-sdk-java#490
This issue was reported in the dropbox java sdk.
To Reproduce
Attempt to call getMetadata
with a filename that has U+2028 in it. It will fail to match the regex and throw java.lang.IllegalArgumentException: String 'path' does not match pattern.
If you attempt to escape the Unicode character, the server returns a malformed path error.
Expected Behavior
Files with a valid path would match the regex and return the file metadata.
Actual Behavior
java.lang.IllegalArgumentException: String 'path' does not match pattern.
or
Malformed path error from the server
Versions
Additional context
I think the fix is as simple as changing the regex to
(/(.|[\r\n\u2028])*|id:.*)|(rev:[0-9a-f]{9,})|(ns:[0-9]+(/.*)?)
in files.stone but would like somebody with more stone experience to chime in.
In the pypi page here. It looks like there has to be some docs. but they aren't.
*Introduction
* Motivation
* Installation
* Language Reference
* Builtin Generators
* Managing Specs
* Evolving a Spec
* Writing a Generator
* JSON Serializer
* Network Protocol
But all results whith the 404 page at pypi.
Before you start
Have you checked StackOverflow, previous issues, and Dropbox Developer Forums for help?
Yes
What is your question?
A clear and concise description the question.
setup.py lists pytest at <5
:
Lines 26 to 28 in 314d297
<7
:Line 3 in 314d297
Just wanted to check which one's correct?
Hi all! This is actually a bug I introduced, just making sure it's tracked somewhere.
Mypy will not correctly detect the type of a
in the following code snippet of a .pyi
file.
a = ... # type: Text
from typing import Text
Mypy will correctly detect the type of a
if we literally just swap those lines.
from typing import Text
a = ... # type: Text
Unfortunately, my python_type_stubs
generator does exactly the incorrect behavior, as shown in this test suite:
stone/test/test_python_type_stubs.py
Line 306 in 4f4e590
The result: Our apiv2 stuff is often poorly typed as typing.Any
s :(
If we look at _generate_base_namespace_module, sure enough the _generate_imports_needed_for_typing
is the final thing called. This is because we build up a list of all the things that, well, do need to be imported.
Just my opinion, feel free to come up with another solution:
self.emit_raw
should not immediately write to a file - perhaps split this into something like self.append_to_output_buffer(...)
, that doesn't actually write anything to a file yet, and a self.emit_output_buffer(...)
. This would let us do something like:
<stone doing its' thing, doing self.append_output_buffer() while figuring out what imports are needed>
self.prepend_output_buffer(the imports required)
self.emit_output_buffer
I'm having trouble running stone python_client
: it seems to require --module-name
and --class-name
, but it seems impossible to actually specify them:
$ stone python_client python dropbox-api-spec/*.stone
usage: python-client-generator [-h] -m MODULE_NAME -c CLASS_NAME
python-client-generator: error: the following arguments are required: -m/--module-name, -c/--class-name
$ stone --module-name dbx --class-name Api python_client python dropbox-api-spec/*.stone
usage: stone [-h] [-v] [--clean-build] [-f FILTER_BY_ROUTE_ATTR]
[-a ATTRIBUTE]
[-w WHITELIST_NAMESPACE_ROUTES | -b BLACKLIST_NAMESPACE_ROUTES]
generator output [spec [spec ...]]
stone: error: unrecognized arguments: --module-name --class-name
# I tried all possible places for specifying the arguments: after all positional args, after python_client etc
# Always fails regardless
+ /usr/bin/python3 -Bm pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.8, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /home/tkloczko/rpmbuild/BUILD/stone-3.2.1
plugins: flaky-3.6.1, forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, asyncio-0.14.0, expect-1.1.0, cov-2.11.1, mock-3.5.1, httpbin-1.0.0, xdist-2.2.1, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, hypothesis-6.8.1, pyfakefs-4.4.0, freezegun-0.4.2
collected 162 items
test/test_backend.py .......... [ 6%]
test/test_cli.py . [ 6%]
test/test_js_client.py ... [ 8%]
test/test_python_client.py ...... [ 12%]
test/test_python_gen.py .............................................................. [ 50%]
test/test_python_type_stubs.py ........ [ 55%]
test/test_python_types.py ..... [ 58%]
test/test_stone.py ......................................... [ 83%]
test/test_stone_internal.py ........ [ 88%]
test/test_stone_route_whitelist.py ....... [ 93%]
test/test_tsd_client.py ... [ 95%]
test/test_tsd_types.py ........ [100%]
============================================================================= warnings summary =============================================================================
test/test_stone.py: 406 warnings
test/test_stone_route_whitelist.py: 35 warnings
/home/tkloczko/rpmbuild/BUILD/stone-3.2.1/stone/frontend/ir_generator.py:1077: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
argspec = inspect.getargspec(data_type_class.__init__) # noqa: E501 # pylint: disable=deprecated-method,useless-suppression
-- Docs: https://docs.pytest.org/en/stable/warnings.html
==================================================================== 162 passed, 441 warnings in 20.13s ====================================================================
We should probably automate verification of everything in the examples
directory.
Hey Stone team! We found an issue with the typing stub code generated for properties. I believe this block of code should be deleted:
stone/stone/backends/python_type_stubs.py
Lines 369 to 371 in e695746
The effect of this block is to always add Optional[]
around the return type of a property getter. (The logic looks backwards, but for nullable and void types there already is an Optional[]
.)
But if you look at the runtime code generated, you'll see that for a non-nullable default-less property, None
is never returned, it raises an exception if the value isn't set:
stone/stone/backends/python_types.py
Lines 625 to 634 in e695746
I will propose a PR to fix this. I have tested the PR in the Dropbox server repo, and it caused only two errors, and those look genuine.
CC: @ilevkivskyi
Is it possible to use lowerCamelCase format for field names for the Python types like the following?
struct Person
firstName String
lastName String
I'm writing a spec for an API with a lowerCamelCase format for field names but the generated Python types for it changed the field names to underscore format.
I'm splitting the Go SDK generator into a types generator and a client generator. To do this, I've introduced a go_helpers
module. But I'm having trouble using the helper when invoking the stone
helper script. Specifically, this does not work:
$ stone -a :all -v "go_client.stoneg.py" "$gen_dir" "$spec_dir"/*.stone
...
error: Importing generator 'go_client.stoneg.py' module raised an exception:
Traceback (most recent call last):
File "/usr/local/bin/stone", line 9, in <module>
load_entry_point('stone==0.1', 'console_scripts', 'stone')()
File "/usr/local/lib/python3.5/site-packages/stone-0.1-py3.5.egg/stone/cli.py", line 303, in main
generator_module = imp.load_source('user_generator', args.generator)
File "/usr/local/Cellar/python3/3.5.2/Frameworks/Python.framework/Versions/3.5/lib/python3.5/imp.py", line 172, in load_source
module = _load(spec)
File "<frozen importlib._bootstrap>", line 693, in _load
File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 665, in exec_module
File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
File "go_client.stoneg.py", line 1, in <module>
from go_helpers import HEADER
ImportError: No module named 'go_helpers'
But this does:
$ python3 -m stone.cli -v go_client.stoneg.py /tmp dropbox-api-spec/*.stone
...
INFO:stone.compiler:Running generator: GoTypesGenerator
Is there a way to indicate dependencies between routes? For example very often, visiting /login
is required before /logout
.
There could also be dependencies between data, let's say you have:
alias UUID = String(pattern="[0-9a-fA-F]{8}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{12}")
struct RegisterReq
"Describes a user registration request, this is often required to do anything else."
username String
"Given username for that user."
struct RegisterResp
"Describes a registration response, indicates a user has registered and is now identifiable"
uuid UUID
"A universal identifier"
struct LogoutReq
"Triggers the user to logout, requires a UUID to have been received"
@Depends(RegisterResp)
uuid UUID
route register_person(RegisterReq, RegisterResp, Void)
route logout_person(LogoutReq, Void, Void)
I think dependency semantics are very useful when testing routes, so if there's no semanics in place right now, I'd like to propose a @Depends
annotation for dependant datatypes, from which we can infer the natural order of requests, used as seen above.
Describe the bug
distutils
is deprecated and slated for removal in Python 3.12
https://peps.python.org/pep-0632//#migration-advice
To Reproduce
Run with Python 3.12
============================= test session starts ==============================
platform linux -- Python 3.12.2, pytest-7.4.4, pluggy-1.4.0
rootdir: /build/source
collected 63 items / 10 errors / 1 deselected / 62 selected
==================================== ERRORS ====================================
____________________ ERROR collecting test/test_backend.py _____________________
ImportError while importing test module '/build/source/test/test_backend.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/nix/store/n3jf1lkdfakxskzsm4nlhss8mdgmcqhc-python3-3.12.2/lib/python3.12/importlib/__init__.py:90: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
test/test_backend.py:7: in <module>
from stone.ir import (
stone/ir/__init__.py:1: in <module>
from .api import * # noqa: F401,F403
stone/ir/api.py:5: in <module>
from distutils.version import StrictVersion
E ModuleNotFoundError: No module named 'distutils'
___________________ ERROR collecting test/test_js_client.py ____________________
[...]
I can think of two ways to fix this:
'long'
with 'int'
hereA potential problem for (1) would be that it will not only change the generated .pyi files but also the generated .py files. I do think that even in Python 2 the distinction between int and long is rarely useful though, so perhaps this is acceptable.
The problem with (2) is that it's a lot more code, since the architecture for the override_dict doesn't really map cleanly to the way is_integer_type() is used. (In order to allow separate mappings for Int32, UIint32, Int64 and UInt64, we'd need 4 isinstance checks.)
Describe the bug
Stone's frontend uses the inspect.getargspec
method which was deprecated in Python 3.0 and removed in 3.11 (see https://docs.python.org/3.10/library/inspect.html#inspect.getargspec).
stone/stone/frontend/ir_generator.py
Line 1077 in bdd49bc
This results in attribute errors when running stone in Python 3.11.
Versions
Additional context
The recommended migration path is to use inspect.getfullargspec
as drop-in replacement.
Please clarify somewhere how Union of Union is serialized. Also, how Union of Struct with subtypes is serialized.
As I undersnand, there should be several ".tag" properties in the same JSON object, which is not so easy to deduct.
This was an oversight on my original implementation of python_type_stubs
- after integrating it into our codebase I've seen a few mypy errors along the lines of
has no attribute 'SomeStoneType_validator
and
Module has no attribute "name_of_a_route"
I might work on it this weekend, just wanted to track as an issue
Describe the bug
This project uses the imp
module which has been deprecated since Python 3.4 and removed in 3.12:
PendingDeprecationWarning
since 3.4 (2014)DeprecationWarning
since 3.5 (2015)DeprecationWarning
to say removal in 3.12 since 3.10 (2021)Python 3.12 is set for release on 2023-10-02 and this library is one of the top 5,000 most-downloaded from PyPI.
Please could you upgrade to use importlib
? The imp
docs have suggestions on what to use to replace each function and constant.
To Reproduce
git grep "\bimp\b"
Expected Behavior
Works on Python 3.12.
Actual Behavior
Will fail due to removed imp module.
Additional context
Add any other context about the problem here.
Line 8 in bdd49bc
Line 178 in bdd49bc
Line 351 in bdd49bc
I have a parameter coming from server which is logically a boolean but not in the form of true
or false
. 1 means true
and 0 means false
. Can I do something like: true_or_false_in_int Boolean = 1
? Since I care about only having a Python backend where assigning 0 or 1 to bool
yields False
or True
When I run using the pypi distfile, I get:
============================================================================== ERRORS ===============================================================================
______________________________________________________________ ERROR collecting test/test_js_client.py ______________________________________________________________
ImportError while importing test module '/scratch/net/py-stone/work/stone-3.3.8/test/test_js_client.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/pkg/lib/python3.12/importlib/__init__.py:90: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
test/test_js_client.py:7: in <module>
from test.backend_test_util import _mock_output
E ModuleNotFoundError: No module named 'test.backend_test_util'
______________________________________________________________ ERROR collecting test/test_tsd_types.py ______________________________________________________________
ImportError while importing test module '/scratch/net/py-stone/work/stone-3.3.8/test/test_tsd_types.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/pkg/lib/python3.12/importlib/__init__.py:90: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
test/test_tsd_types.py:19: in <module>
from test.backend_test_util import _mock_output
E ModuleNotFoundError: No module named 'test.backend_test_util'
====================================================================== short test summary info ======================================================================
ERROR test/test_js_client.py
ERROR test/test_tsd_types.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I only find two matches for the string:
stone-3.3.8/test/test_js_client.py:from test.backend_test_util import _mock_output
stone-3.3.8/test/test_tsd_types.py:from test.backend_test_util import _mock_output
I'm not sure where this should be coming from, did I miss installing a dependency?
Before you start
Have you checked StackOverflow, previous issues, and Dropbox Developer Forums for help?
Yes
What is your question?
A clear and concise description of the question.
I'm currently in the process of updating the dropbox-sdk-python on conda-forge which seems to require stone: https://github.com/dropbox/dropbox-sdk-python/blob/250b116f77b8f22351c1e2e0eb625d0d19f86f2c/setup.py#L30
I, therefore, wanted to bring stone to conda-forge as well, but the tests fail for windows, as can be seen in the logs here: https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=277449&view=logs&j=240f1fee-52bc-5498-a14a-8361bde76ba0&t=23010c01-6a9d-5f94-9037-792618644b77
From a cursory overview, I can also see that you're not testing on windows (Although I just quickly checked that, not sure if you're testing windows differently?):
stone/.github/workflows/CI.yaml
Lines 11 to 17 in 314d297
Is it correct then, that stone is only available on macOS and Linux, or am I missing something (Which I would assume since the python client, which depends on stone, is)?
Screenshots
If applicable, add screenshots to help explain your question.
Versions
Additional context
Add any other context about the question here.
The PR is here: conda-forge/staged-recipes#13960
Edit: The test logs also suggest only macOS, from what I could find: https://github.com/dropbox/stone/actions/runs/514261108
Describe the bug
Building from source fails with
Traceback (most recent call last):
File "/private/tmp/pkgsrc/net/py-stone/work/stone-3.3.1/setup.py", line 21, in <module>
with open('requirements.txt') as f: # pylint: disable=W1514
FileNotFoundError: [Errno 2] No such file or directory: 'requirements.txt'
To Reproduce
Get source code from PyPI, and build.
Expected Behavior
Builds
Actual Behavior
Does not build
Can I define generic types in stone? Response wrappers like Result<T>
PEP 484 intentionally recommends using int/float instead of the numeric ABCs, and mypy doesn't like them at all:
from numbers import Integral, Real
def f(i: Integral, x: Real) -> Real:
return i * x
f(1, 3.14)
gives these errors:
__tmp__.py:4: error: Argument 1 to "f" has incompatible type "int"; expected "Integral"
__tmp__.py:4: error: Argument 2 to "f" has incompatible type "float"; expected "Real"
Currently, when targeting Python, Stone does not add #type
declarations to generated code. I don't know how difficult this would be, but it probably should.
This project still uses pytest-runner, but the upstream repository is archived:
Deprecation Notice
pytest-runner depends on deprecated features of setuptools and relies on features that break security mechanisms in pip. For example 'setup_requires' and 'tests_require' bypass pip --require-hashes. See also https://github.com/pypa/setuptools/issues/1684.
It is recommended that you:
Remove 'pytest-runner' from your setup_requires, preferably removing the setup_requires option.
Remove 'pytest' and any other testing requirements from tests_require, preferably removing the tests_requires option.
Select a tool to bootstrap and then run tests such as tox.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.