Describe the bug
dropbox/dropbox-sdk-java#490
This issue was reported in the dropbox java sdk.

To Reproduce
Attempt to call getMetadata with a filename that has U+2028 in it. It will fail to match the regex and throw java.lang.IllegalArgumentException: String 'path' does not match pattern.

If you attempt to escape the Unicode character, the server returns a malformed path error.

Expected Behavior
Files with a valid path would match the regex and return the file metadata.

Actual Behavior
java.lang.IllegalArgumentException: String 'path' does not match pattern.
or
Malformed path error from the server

Versions

• What version of the Stone are you using? Latest stone, latest api spec from Sep 1, 2022
• What version of the language are you using? Java
• What platform are you using? (if applicable) Dropbox Java Sdk

Additional context
I think the fix is as simple as changing the regex to
(/(.|[\r\n\u2028])*|id:.*)|(rev:[0-9a-f]{9,})|(ns:[0-9]+(/.*)?)
in files.stone but would like somebody with more stone experience to chime in.

I'm getting the following when running ./mypy-run.sh as of 2e04e96:

stone/target/obj_c_types.py: note: In class "ObjCTypesGenerator":
stone/target/obj_c_types.py:99: error: Need type annotation for variable
stone/target/obj_c_client.py: note: In class "ObjCGenerator":
stone/target/obj_c_client.py:101: error: Need type annotation for variable

These didn't appear in 8fbb9e6.

I'm splitting the Go SDK generator into a types generator and a client generator. To do this, I've introduced a go_helpers module. But I'm having trouble using the helper when invoking the stone helper script. Specifically, this does not work:

$ stone -a :all -v "go_client.stoneg.py" "$gen_dir" "$spec_dir"/*.stone
...
error: Importing generator 'go_client.stoneg.py' module raised an exception:
Traceback (most recent call last):
  File "/usr/local/bin/stone", line 9, in <module>
    load_entry_point('stone==0.1', 'console_scripts', 'stone')()
  File "/usr/local/lib/python3.5/site-packages/stone-0.1-py3.5.egg/stone/cli.py", line 303, in main
    generator_module = imp.load_source('user_generator', args.generator)
  File "/usr/local/Cellar/python3/3.5.2/Frameworks/Python.framework/Versions/3.5/lib/python3.5/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 693, in _load
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "go_client.stoneg.py", line 1, in <module>
    from go_helpers import HEADER
ImportError: No module named 'go_helpers'

But this does:

$ python3 -m stone.cli -v go_client.stoneg.py /tmp dropbox-api-spec/*.stone
...
INFO:stone.compiler:Running generator: GoTypesGenerator

In the pypi page here. It looks like there has to be some docs. but they aren't.

*Introduction
    * Motivation
    * Installation
* Language Reference
* Builtin Generators
* Managing Specs
* Evolving a Spec
* Writing a Generator
* JSON Serializer
* Network Protocol

But all results whith the 404 page at pypi.

Before you start
Have you checked StackOverflow, previous issues, and Dropbox Developer Forums for help?
Yes

What is your question?
A clear and concise description the question.

setup.py lists pytest at <5:

Just wanted to check which one's correct?

PEP 484 intentionally recommends using int/float instead of the numeric ABCs, and mypy doesn't like them at all:

from numbers import Integral, Real
def f(i: Integral, x: Real) -> Real:
    return i * x
f(1, 3.14)

gives these errors:

__tmp__.py:4: error: Argument 1 to "f" has incompatible type "int"; expected "Integral"
__tmp__.py:4: error: Argument 2 to "f" has incompatible type "float"; expected "Real"

Please clarify somewhere how Union of Union is serialized. Also, how Union of Struct with subtypes is serialized.
As I undersnand, there should be several ".tag" properties in the same JSON object, which is not so easy to deduct.

I'm having trouble running stone python_client: it seems to require --module-name and --class-name, but it seems impossible to actually specify them:

$ stone python_client python dropbox-api-spec/*.stone
usage: python-client-generator [-h] -m MODULE_NAME -c CLASS_NAME
python-client-generator: error: the following arguments are required: -m/--module-name, -c/--class-name

$ stone --module-name dbx --class-name Api python_client python dropbox-api-spec/*.stone
usage: stone [-h] [-v] [--clean-build] [-f FILTER_BY_ROUTE_ATTR]
             [-a ATTRIBUTE]
             [-w WHITELIST_NAMESPACE_ROUTES | -b BLACKLIST_NAMESPACE_ROUTES]
             generator output [spec [spec ...]]
stone: error: unrecognized arguments: --module-name --class-name

# I tried all possible places for specifying the arguments: after all positional args, after python_client etc
# Always fails regardless

Describe the bug
Building from source fails with

Traceback (most recent call last):
  File "/private/tmp/pkgsrc/net/py-stone/work/stone-3.3.1/setup.py", line 21, in <module>
    with open('requirements.txt') as f:  # pylint: disable=W1514
FileNotFoundError: [Errno 2] No such file or directory: 'requirements.txt'

To Reproduce
Get source code from PyPI, and build.

Expected Behavior
Builds

Actual Behavior
Does not build

Describe the bug

This project uses the imp module which has been deprecated since Python 3.4 and removed in 3.12:

• Raised PendingDeprecationWarning since 3.4 (2014)
• Raised DeprecationWarning since 3.5 (2015)
• Updated DeprecationWarning to say removal in 3.12 since 3.10 (2021)
• Removal planned for 3.12 (2023)

Python 3.12 is set for release on 2023-10-02 and this library is one of the top 5,000 most-downloaded from PyPI.

Please could you upgrade to use importlib? The imp docs have suggestions on what to use to replace each function and constant.

To Reproduce

git grep "\bimp\b"

Expected Behavior

Works on Python 3.12.

Actual Behavior

Will fail due to removed imp module.

Additional context
Add any other context about the problem here.

Before you start
Have you checked StackOverflow, previous issues, and Dropbox Developer Forums for help?

Yes

What is your question?
A clear and concise description of the question.

I'm currently in the process of updating the dropbox-sdk-python on conda-forge which seems to require stone: https://github.com/dropbox/dropbox-sdk-python/blob/250b116f77b8f22351c1e2e0eb625d0d19f86f2c/setup.py#L30

I, therefore, wanted to bring stone to conda-forge as well, but the tests fail for windows, as can be seen in the logs here: https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=277449&view=logs&j=240f1fee-52bc-5498-a14a-8361bde76ba0&t=23010c01-6a9d-5f94-9037-792618644b77

From a cursory overview, I can also see that you're not testing on windows (Although I just quickly checked that, not sure if you're testing windows differently?):

Is it correct then, that stone is only available on macOS and Linux, or am I missing something (Which I would assume since the python client, which depends on stone, is)?

Screenshots
If applicable, add screenshots to help explain your question.

Versions

• What version of Stone are you using? 3.2.1
• What version of the language are you using?
• What platform are you using? (if applicable) Azure DevOps Pipelines Windows

Additional context
Add any other context about the question here.

The PR is here: conda-forge/staged-recipes#13960

Edit: The test logs also suggest only macOS, from what I could find: https://github.com/dropbox/stone/actions/runs/514261108

Hey Stone team! We found an issue with the typing stub code generated for properties. I believe this block of code should be deleted:

The effect of this block is to always add Optional[] around the return type of a property getter. (The logic looks backwards, but for nullable and void types there already is an Optional[].)

But if you look at the runtime code generated, you'll see that for a non-nullable default-less property, None is never returned, it raises an exception if the value isn't set:

I will propose a PR to fix this. I have tested the PR in the Dropbox server repo, and it caused only two errors, and those look genuine.

CC: @ilevkivskyi

Describe the bug
distutils is deprecated and slated for removal in Python 3.12

https://peps.python.org/pep-0632//#migration-advice

To Reproduce
Run with Python 3.12

============================= test session starts ==============================
platform linux -- Python 3.12.2, pytest-7.4.4, pluggy-1.4.0
rootdir: /build/source
collected 63 items / 10 errors / 1 deselected / 62 selected                    

==================================== ERRORS ====================================
____________________ ERROR collecting test/test_backend.py _____________________
ImportError while importing test module '/build/source/test/test_backend.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/nix/store/n3jf1lkdfakxskzsm4nlhss8mdgmcqhc-python3-3.12.2/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/test_backend.py:7: in <module>
    from stone.ir import (
stone/ir/__init__.py:1: in <module>
    from .api import *  # noqa: F401,F403
stone/ir/api.py:5: in <module>
    from distutils.version import StrictVersion
E   ModuleNotFoundError: No module named 'distutils'
___________________ ERROR collecting test/test_js_client.py ____________________
[...]

Can I define generic types in stone? Response wrappers like Result<T>

We should probably automate verification of everything in the examples directory.

Hi all! This is actually a bug I introduced, just making sure it's tracked somewhere.

The issue

Mypy will not correctly detect the type of a in the following code snippet of a .pyi file.

a = ...  # type: Text
from typing import Text

Mypy will correctly detect the type of a if we literally just swap those lines.

from typing import Text
a = ...  # type: Text

Unfortunately, my python_type_stubs generator does exactly the incorrect behavior, as shown in this test suite:

The result: Our apiv2 stuff is often poorly typed as typing.Anys :(

Where does the issue occur?

If we look at _generate_base_namespace_module, sure enough the _generate_imports_needed_for_typing is the final thing called. This is because we build up a list of all the things that, well, do need to be imported.

The solution

Just my opinion, feel free to come up with another solution:

self.emit_raw should not immediately write to a file - perhaps split this into something like self.append_to_output_buffer(...), that doesn't actually write anything to a file yet, and a self.emit_output_buffer(...). This would let us do something like:

<stone doing its' thing, doing self.append_output_buffer() while figuring out what imports are needed>
self.prepend_output_buffer(the imports required)
self.emit_output_buffer

This was an oversight on my original implementation of python_type_stubs - after integrating it into our codebase I've seen a few mypy errors along the lines of

has no attribute 'SomeStoneType_validator

and

Module has no attribute "name_of_a_route"

I might work on it this weekend, just wanted to track as an issue

Is there a way to indicate dependencies between routes? For example very often, visiting /login is required before /logout.

There could also be dependencies between data, let's say you have:

alias UUID = String(pattern="[0-9a-fA-F]{8}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{4}\-[0-9a-fA-F]{12}")

struct RegisterReq
    "Describes a user registration request, this is often required to do anything else."

    username String
        "Given username for that user."

struct RegisterResp
    "Describes a registration response, indicates a user has registered and is now identifiable"

    uuid UUID
        "A universal identifier"

struct LogoutReq
    "Triggers the user to logout, requires a UUID to have been received"

    @Depends(RegisterResp)
    uuid UUID

route register_person(RegisterReq, RegisterResp, Void)
route logout_person(LogoutReq, Void, Void)

I think dependency semantics are very useful when testing routes, so if there's no semanics in place right now, I'd like to propose a @Depends annotation for dependant datatypes, from which we can infer the natural order of requests, used as seen above.

Currently, when targeting Python, Stone does not add #type declarations to generated code. I don't know how difficult this would be, but it probably should.

I can think of two ways to fix this:

1. The easy way: replace 'long' with 'int' here
2. Do something with the override_dict

A potential problem for (1) would be that it will not only change the generated .pyi files but also the generated .py files. I do think that even in Python 2 the distinction between int and long is rarely useful though, so perhaps this is acceptable.

The problem with (2) is that it's a lot more code, since the architecture for the override_dict doesn't really map cleanly to the way is_integer_type() is used. (In order to allow separate mappings for Int32, UIint32, Int64 and UInt64, we'd need 4 isinstance checks.)

Describe the bug
Stone's frontend uses the inspect.getargspec method which was deprecated in Python 3.0 and removed in 3.11 (see https://docs.python.org/3.10/library/inspect.html#inspect.getargspec).

This results in attribute errors when running stone in Python 3.11.

Versions

• What version of the Stone are you using? 3.3.1
• What version of the language are you using? Python 3.11
• What platform are you using? macOS 13

Additional context
The recommended migration path is to use inspect.getfullargspec as drop-in replacement.

Is it possible to use lowerCamelCase format for field names for the Python types like the following?

struct Person
    firstName String
    lastName String

I'm writing a spec for an API with a lowerCamelCase format for field names but the generated Python types for it changed the field names to underscore format.

I have a parameter coming from server which is logically a boolean but not in the form of true or false. 1 means true and 0 means false. Can I do something like: true_or_false_in_int Boolean = 1? Since I care about only having a Python backend where assigning 0 or 1 to bool yields False or True