Giter VIP home page Giter VIP logo

langkit's Introduction

Langkit

Langkit (nickname for language kit) is a tool whose purpose is to make it easy to create syntactic and semantic analysis engines. Write a language specification in our Python DSL and Langkit will generate for you an Ada library with bindings for the C and Python programming languages.

The generated library is meant to provide a basis to write tooling, including tools working on potentially changing and incorrect code, such as IDEs.

The currently main Langkit user is Libadalang, a high performance semantic engine for the Ada programming language.

Dependencies

To use Langkit, you will need:

  • A Python 3.9 interpreter (or more recent). Python2 is no longer supported.
  • Some Python libraries, including the Mako template system for Python (see requirements-pypi.txt and requirements-github.txt for the full list).
  • A recent version of the GNAT Ada compiler, either from your OS's packages, or use Alire to get one.
  • The gnatcoll-core library.
  • Ada bindings for GMP and Libiconv, from gnatcoll-bindings.
  • The VSS library.
  • The Prettier-Ada library.
  • The AdaSAT library.

For all Ada libraries (GNATcoll, VSS, Prettier-Ada, AdaSAT), make sure to install the version that corresponds to the version of Langkit that is built. For instance, build all 24.1 branches, or all master branches. Mixing versions is not supported.

Install

We assume below that all the Ada dependencies are installed under the $PREFIX directory, and that the environment is properly set up to use it:

  • GPRbuild has access to project files ($PREFIX/share/gpr is in GPR_PROJECT_PATH, for example).

  • The dynamic linker has access to shared libraries ($PREFIX/lib is in LD_LIBRARY_PATH, for exmaple).

First, clone the adasat repository in the langkit subdirectory of the langkit repository:

$ (cd langkit; git clone https://github.com/AdaCore/adasat)

Then, install the Langkit Python package itself:

$ pip install .

Build the Libpythonlang and Liblktlang support libraries:

$ python manage.py make --no-mypy --library-types=static,static-pic,relocatable

Install the Langkit_Support library:

$ python manage.py install-langkit-support $PREFIX --library-types=static,static-pic,relocatable

Install the Libpythonlang and Liblktlang support libraries:

$ (cd contrib/python && ./manage.py install $PREFIX --library-types=static,static-pic,relocatable --disable-all-mains)
$ (cd contrib/lkt && ./manage.py install $PREFIX --library-types=static,static-pic,relocatable --disable-all-mains)
$ pip install contrib/python/build/python
$ pip install contrib/lkt/build/python

If you are interested in shared (relocatable) libraries only, you can omit the --library-types arguments.

Testing

In order to run the testsuite, launch the following command from the top-level directory:

$ python manage.py test

This is just a wrapper passing convenient options to the real testsuite driver that is in testsuite/testsuite.py.

Note that even though the testsuite framework requires Python 3.9, it is possible to run the tests themselves using a different Python interpreter. For instance, to run them using Python 3.7, run:

$ python manage.py test --with-python=python3.7

If you want to learn more about this test driver's options (for instance to run tests under Valgrind), add a -h flag.

Documentation

The developer and user's documentation for Langkit is in langkit/doc. You can consult it as a text files or you can build it. For instance, to generate HTML documents, run from the top directory:

$ make -C doc html

And then open the following file in your favorite browser:

doc/_build/html/index.html

Bootstrapping a new language engine

Nothing is more simple than getting an initial project skeleton to work on a new language engine. Imagine you want to create an engine for the Foo language, run from the top-level directory:

$ python scripts/create-project.py Foo

And then have a look at the created foo directory: you have minimal lexers and parsers and a manage.py script you can use to build this new engine:

$ python foo/manage.py make

Here you are!

Developer tools

Langkit uses mako templates generating Ada, C and Python code. This can be hard to read. To ease development, Vim syntax files are available under the utils directory (see makoada.vim, makocpp.vim). Install them in your $HOME/.vim/syntax directory to get automatic highlighting of the template files.

langkit's People

Contributors

adadoom3 avatar anisimkov avatar asarhaddon avatar atupone avatar bobduff avatar briot avatar danielmercier avatar eliericha avatar geoffreycopin avatar godunko avatar hugogguerrier avatar joaopsazevedo avatar leocreuse avatar lordaro avatar miranda-adacore avatar nikokrock avatar okellogg avatar pmderodat avatar raph-amiard avatar reznikmm avatar roldak avatar ronan-d avatar sad-poet avatar setton avatar stephe-ada-guru avatar thvnx avatar turbogit avatar yakobowski avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

langkit's Issues

GNAT GPL 2016 encounters a GNAT BUG

Generating source for libgprlang ...
Generation complete!
Building the generated source code ...
gcc -c -fPIC -gnatwa -gnatyg -g -O0 -gnatwe -gnata -gnatef langkit_support-adalog-main_support.ads
+===========================GNAT BUG DETECTED==============================+
| GPL 2016 (20160515-49) (x86_64-pc-linux-gnu) GCC error: |
| in process_type, at ada/gcc-interface/trans.c:9616 |
| Error detected at langkit_support-adalog-unify_one_side.ads:105:9 [langkit_support-adalog-unify.ads:51:4 [langkit_support-adalog-eq_same.ads:33:4 [langkit_support-adalog-main_support.ads:8:4]]]|
| Please submit a bug report by email to [email protected]. |
| GAP members can alternatively use GNAT Tracker: |
| http://www.adacore.com/ section 'send a report'. |
| See gnatinfo.txt for full info on procedure for submitting bugs. |
| Use a subject line meaningful to you and us to track the bug. |
| Include the entire contents of this bug box in the report. |
| Include the exact command that you entered. |
| Also include sources listed below. |
| Use plain ASCII or MIME attachment(s). |
+==========================================================================+

I ran git bisect; it reports 91cf97e is the first bad commit

commit 91cf97e
Author: Pierre-Marie de Rodat [email protected]
Date: Mon Feb 13 18:05:58 2017 +0100

astdoc: add documentation for structure types

TN: Q213-023

I don't understand the code well enough to investigate further

Struggles building Langkit with Alire

Bonsoir, hope that you're all doing well.

I am trying to evaluate langkit, aiming to simplify
the development of an LSP and a set of linters/formaters for a DSL. I struggled
getting it to build, here is a lengthy summary of what I encountered and where I stopped.

The documentation recommends GNAT Community 2021.
As I was surprised that this was the latest release of GNAT, I found the blogs
mentioning alire being the recommended way to get gnat, so I gave it a try that way, on macOS.

I did a manual setup of alire for langkit and added a dependency to
langkit_support, trying to avoid having to build it.

I would have a hard time bringing the entire Ada toolchain to our devenvs,
and aimed to build as little as possible, hopefully requesting a small set to
be released (in pip using a bdist ? <3) if the eval was successful.

In the created alire.toml :

[[depends-on]]
langkit_support = "^24.0.0"

I realized that default compilation with manage.py was still trying to compile
langkit_support, I found the --no-langkit-support option to bypass it.

Suggestion : Separating langkit_support out of the dependency list might be justified given
that it's expected to be built.

Moving forward I was stumbling upon an error caused by xmlada_shared.gpr not
found from the xmlada lib. I was using the one brought in by langkit_support.

xmlada_unicode.gprxmlada_unicode.gpr:24:06: imported project file "../xmlada_shared.gpr" not found
xmlada_unicode.gpr:24:06: imported by "/Users/bourgeoisclement/.local/share/alire/builds/xmlada_24.0.0_ae5a015b/b0131ab91b91aa28704f2c86cc19d71588851835808f51b4132600c1599b3c69/unicode/xmlada_unicode.gpr"
xmlada_unicode.gpr:24:06: imported by "/Users/bourgeoisclement/.local/share/alire/builds/xmlada_24.0.0_ae5a015b/b0131ab91b91aa28704f2c86cc19d71588851835808f51b4132600c1599b3c69/distrib/xmlada.gpr"
xmlada_unicode.gpr:24:06: imported by "/Users/bourgeoisclement/.local/share/alire/builds/libgpr_24.0.0_d9c96bda/9098d495aeffac6e11d3d93c6b8c5c10b8181054d2fc132f3bf2a0f70303aa83/gpr/gpr.gpr"
xmlada_unicode.gpr:24:06: imported by "/Users/bourgeoisclement/.local/share/alire/builds/gnatcoll_24.0.0_11c512d1/47cf7d7e8a2b7b3c0e9536d64ef2252d7e51940df5bfe09d9251db5741773a95/gnatcoll.gpr"
xmlada_unicode.gpr:24:06: imported by "/Users/bourgeoisclement/repos/langkit/contrib/lkt/build/liblktlang.gpr"
xmlada_unicode.gpr:29:25: unknown package or project "Xmlada_Shared"
xmlada_unicode.gpr:29:25: wrong expression kind for attribute "library_kind"

That most likely is a problem to report to xmlada, and the way xmlada_24.0.0 is
packaged for alire.
I fixed it by running ./configure in the xmlada GPR_PROJECT_PATH location to generate the
Xmlada_Shared.gpr file. Not langkit fault I know :)

Next I ran into this :

python manage.py make --no-langkit-support

Generating source for libpythonlang...
Generating source for liblktlang...
Generation complete!
Building the generated source code
Building for config (BuildMode.prod, LibraryType.static)
gpr.gpr:26:04: value "static" is illegal for typed string "bld"

I figured out that this was caused by a conflict in the supported GPR_BUILD
values in gpr.gpr, I forced that to be "production" in the gpr file, not sure which
part of the coupling should be the source of truth, but I wanted to move
forward.

After the above, the prettier_ada dependency reported being missing, that was not mentionned in the
intro, so worth updating, I assume that it's pretty recent.
I cloned the repository, setup an alire project there as well with vss and got that solved.

Finally I blocked on :

   [Ada]          libpythonlang_support-adalog-main_support.ads
/Users/bourgeoisclement/repos/langkit/contrib/python/build/src/libpythonlang_support-adalog-solver.adb:1595:36: error: "Free" is not a visible entity of "Libpythonlang_AdaSAT"

   compilation of libpythonlang_support-adalog-main_support.ads failed

gprbuild: *** compilation phase failed
Build failed: error while running gprbuild -p -j4
-P/Users/bourgeoisclement/repos/langkit/contrib/python/build/libpythonlang.gpr
-XBUILD_MODE=prod -XLIBRARY_TYPE=static -XGPR_BUILD=static -XXMLADA_BUILD=static
-XLIBPYTHONLANG_WARNINGS=true -gnatef:

I assume that it was foolish to expect langkit_support to work on master, so I
went to 24.0 but it failed similarly.

Got this error on the release branch that did not stop the build from progressing
but still sharing it for visiblity :

Generating source for liblktlang...
/Users/bourgeoisclement/repos/langkit/contrib/lkt/language/parser.py:1248:1: Then expression should have a default value provided, in cases where the type of the provided function's return type (here Int) does not have a default null value.
Errors, exiting
Generation complete!

Retried without trying to use the langkit_support from alire, building it directly, but got a similar
error :

   [Ada]          langkit_support-adalog-solver.adb
langkit_support-adalog-solver.adb:1595:22: error: "Free" is not a visible entity of "AdaSAT"

   compilation of langkit_support-adalog-solver.adb failed

Should I use an older gnat ? I was using 13.2.0.

Final note, manage.py seems to run twice, always retrying after a failure, not a big deal but that was surprising.

Let me know some of my notes are unclear or if you need more context.

Testsuite is using yaml.load CVE-2017-18342

In PyYAML before 4.1, the yaml.load() API could execute arbitrary code. In other words, yaml.safe_load is not used.
I don't know the status after 4.1, however in gentoo with pyyaml 5.1 I cannot run test for the yaml.load() use

Please support LDFLAGS

This is mainly triggered by use on macOS via Alire.

On macOS, we have to use relocatable builds because static SALs don’t work (gprbuild issue 97), and this means that we need to provide libgmp.dylib.

The macOS compilers available via Alire don’t search /usr/local by default, which means that we have to tell gprbuild where to look, and the only obvious mechanism is via LDFLAGS.

Please see also libadalang issue 958.

README:setup, doc

README.md doesn’t mention the need to call setup.py, or how to call it (the documentation suggests using setup.py/easy_install/pip/..., whatever ... is, but we’ve already installed REQUIREMENTS.dev).

Also, it says the documentation is found in langkit/doc, but it’s actually in doc.

People might get started quicker if eval $(./manage.py setenv) was mentioned?

Errors during test

During test phase I got a lot of
ModuleNotFoundError: No module named 'liblktlang'

I cannot find from where this module come

Missing check in Bind

Found that by writing the following property in GenericInstantiation:

    xref_equation = Property(
        Bind(Entity.generic_entity_name, Entity.designated_generic_decl)
    )

The first param should denote a logic variable, but denotes a T.Name.Entity, and no error is issued

Thank PM :)

Fail to build with --library-types=static-pic

I am building langkit on Linux using the following command:

python manage.py build-langkit-support --library-types=static-pic

I get the following linking error:

/usr/bin/ld: -r and -pie may not be used together
collect2: error: ld returned 1 exit status
gprlib: call to linker driver /usr/bin/gcc failed
gprbuild: could not build library for project langkit_support
Traceback (most recent call last):
  File "/home/tim/projects/aur/ada_language_server/src/langkit/manage.py", line 291, in <module>
    args.func(args, unknown_args)
  File "/home/tim/projects/aur/ada_language_server/src/langkit/manage.py", line 87, in wrapper
    fn(args)
  File "/home/tim/projects/aur/ada_language_server/src/langkit/manage.py", line 118, in build_langkit_support
    subprocess.check_call(base_argv + [f"-XLIBRARY_TYPE={library_type}"])
  File "/usr/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['gprbuild', '-P', '/home/tim/projects/aur/ada_language_server/src/langkit/support/langkit_support.gpr', '-p', '-j16', '-XBUILD_MODE=dev', '-XLIBRARY_TYPE=static-pic']' returned non-zero exit status 4.
==> ERROR: A failure occurred in build().
    Aborting...

GIT message

I am using the latest version of Gnat/Ada, and when I compile and run the code I get an error:
[2020-06-06 16:09:45] Could not locate executable on path: git

The project settings specify Version Control: None

Do I have something incorrectly set, and/or is there a way to suppress this message?

langkit references gnatcoll_iconv, gnatcoll-bindings currently provides gnatcoll-iconv.gpr

Currently libadalang (which includes langkit) fails to build on new gnatcoll.

Source of failure appears to be ./langkit/templates/project_file.mako line: 6. If "gnatcoll_iconv" is changed to "gnat-iconv" then libadalang builds successfully, at least for me.

Alternate workaround was to put symbolic link(s) in <install_dir>/share/gpr if this falls into the category of waiting for gnatcoll to stabilize.

Error compiling Ada code when using build-langkit-support option

I have cloned the repository via:
git clone https://github.com/AdaCore/langkit.git
and then followed the instructions to build the code. I typed in:
pip install .
and this worked fine. I then typed in:
python manage.py build-langkit-support --library-types=static,static-pic,relocatable
and the following set of error messages are displayed:

error: "langkit_support-adalog-abstract_relation.adb" must be recompiled ("langkit_support-adalog.ads" has been modified)
error: "langkit_support-adalog-debug.adb" must be recompiled ("langkit_support-adalog.ads" has been modified)
error: "langkit_support-adalog-eq_same.adb" must be recompiled ("langkit_support-adalog.ads" has been modified)
error: "langkit_support-adalog-logic_ref.adb" must be recompiled ("langkit_support-adalog.ads" has been modified)
error: "langkit_support-adalog-logic_var.ads" must be recompiled ("langkit_support-adalog.ads" has been modified)
error: "langkit_support-adalog-main_support.adb" must be recompiled ("langkit_support-adalog.ads" has been modified)
error: "langkit_support-adalog-operations.adb" must be recompiled ("langkit_support-adalog.ads" has been modified)
error: "langkit_support-adalog-predicates.adb" must be recompiled ("langkit_support-adalog.ads" has been modified)
error: "langkit_support-adalog-pure_relations.adb" must be recompiled ("langkit_support-adalog.ads" has been modified)
error: "langkit_support-adalog-relations.adb" must be recompiled ("langkit_support-adalog.ads" has been modified)
error: "langkit_support-adalog-unify.adb" must be recompiled ("langkit_support-adalog.ads" has been modified)
error: "langkit_support-adalog-unify_lr.adb" must be recompiled ("langkit_support-adalog.ads" has been modified)
error: "langkit_support-adalog-unify_one_side.adb" must be recompiled ("langkit_support-adalog.ads" has been modified)
error: "langkit_support-adalog.ads" has been modified and must be recompiled
error: "langkit_support-diagnostics-output.adb" must be recompiled ("langkit_support-diagnostics.ads" has been modified)
error: "langkit_support-diagnostics.adb" must be recompiled ("langkit_support-diagnostics.ads" has been modified)
error: "langkit_support-errors.ads" has been modified and must be recompiled
error: "langkit_support-file_readers.adb" must be recompiled ("langkit_support-diagnostics.ads" has been modified)
error: "langkit_support-generic_api-analysis.adb" must be recompiled ("langkit_support-diagnostics.ads" has been modified)
error: "langkit_support-generic_api.adb" must be recompiled ("langkit_support-diagnostics.ads" has been modified)
error: "langkit_support-internal.ads" must be recompiled ("langkit_support-diagnostics.ads" has been modified)
error: "langkit_support-lexical_envs.ads" has been modified and must be recompiled
error: "langkit_support-lexical_envs_impl.adb" must be recompiled ("langkit_support-errors.ads" has been modified)
error: "langkit_support-slocs.adb" must be recompiled ("langkit_support-slocs.ads" has been modified)
error: "langkit_support-symbols-precomputed.adb" must be recompiled ("langkit_support-symbols.ads" has been modified)
error: "langkit_support-symbols.adb" must be recompiled ("langkit_support-symbols.ads" has been modified)
error: "langkit_support-text.adb" must be recompiled ("langkit_support-text.ads" has been modified)
error: "langkit_support-token_data_handlers.adb" must be recompiled ("langkit_support-slocs.ads" has been modified)
gprlib: invocation of /opt/ides/gnat/2020/bin/gnatbind failed
gprbuild: could not build library for project langkit_support
Traceback (most recent call last):
  File "/home/xxx/Programs/Development-Tools/GNAT/langkit/manage.py", line 291, in <module>
    args.func(args, unknown_args)
  File "/home/xxx/Programs/Development-Tools/GNAT/langkit/manage.py", line 87, in wrapper
    fn(args)
  File "/home/xxx/Programs/Development-Tools/GNAT/langkit/manage.py", line 118, in build_langkit_support
    subprocess.check_call(base_argv + [f"-XLIBRARY_TYPE={library_type}"])
  File "/usr/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['gprbuild', '-P', '/home/xxx/Programs/Development-Tools/GNAT/langkit/support/langkit_support.gpr', '-p', '-j4', '-XBUILD_MODE=dev', '-XLIBRARY_TYPE=relocatable']' returned non-zero exit status 4.

I have tried the command without the --library-types to build only one library type, but the error message is the same. What am I doing wrong ?

Missing Value of SLOC

Dear adaCore,

I am missing some basic functionality.
The Value function (the inverse of image )
is not present.

A simple solution might be to add functions like

   function Source_Location_Value (Source_Location_String : String) return Source_Location
   is
      Index_Colon : constant Natural := Index (Source_Location_String, ":");
      Line : constant Line_Number := 
        Line_Number'Value (Head (Source_Location_String, Index_Colon - Source_Location_String'First));
      Column : constant Column_Number := 
        Column_Number'Value (Tail (Source_Location_String, Source_Location_String'Last - Index_Colon));
   begin
      return (Line => Line,
              Column => Column);
   end Source_Location_Value;

   function Source_Location_Range_Value (Source_Location_Range_String : String) 
                                         return Source_Location_Range
   is
      Index_Dash : constant Natural := Index (Source_Location_Range_String, "-");
      Start_Location_String : constant String := 
        Head (Source_Location_Range_String, Index_Dash - Source_Location_Range_String'First);
      Start_Location : constant Source_Location := Source_Location_Value (Start_Location_String);
      End_Location_String : constant String := 
        Tail (Source_Location_Range_String, Source_Location_Range_String'Last - Index_Dash);
      End_Location : constant Source_Location := Source_Location_Value (End_Location_String);
   begin
      return (Start_Line   => Start_Location.Line,
              End_Line     => End_Location.Line,
              Start_Column => Start_Location.Column, 
              End_Column   => End_Location.Column);
   end Source_Location_Range_Value;

Free functions

As discussed live, opening an issue for this.

For the moment we can only have properties, that apply to ast nodes. Properties are akin to methods. As a consequence, we have a lot of code that should be in functions/static methods and that in the end is on nodes.

This issue is to implement free functions in the DSL. This encompasses generating code for them, as well as making them part of the public API.

Don't copy extensions when building

In a similar fashion to what we do for mains, it would be desirable to not copy language specific extension packages when building, but rather reference the extension directory directly.

This would make those files more tool friendly (GPS for example)

libadalang test runner on Windows with Python3.8+ cannot find dlls

Seems that Python changed how it handles dll dependencies in 3.8, and now extra paths need to be added explicitly:
https://docs.python.org/3/library/os.html#os.add_dll_directory

Currently it fails with a message something along the lines of:

Traceback (most recent call last):
  File "C:\libadalang\tmp\name_resolution__S709-005\test.py", line 1, in <module>
    import libadalang as lal
  File "C:\libadalang\build\python\libadalang\__init__.py", line 94, in <module>
    _c_lib = ctypes.cdll.LoadLibrary(_c_lib_path)
  File "C:\Python310\lib\ctypes\__init__.py", line 452, in LoadLibrary
    return self._dlltype(name)
  File "C:\Python310\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'libadalang.dll' (or one of its dependencies). Try using the full path with constructor syntax.

As a cheap and nasty hack, the following in module_py.mako just above the LoadLibrary call seems to work, but could probably be improved:

for path in os.environ['PATH'].split(';'):
    if os.path.exists(path):
        os.add_dll_directory(path)

_c_lib = ctypes.cdll.LoadLibrary(_c_lib_path)

Diagnostics.print_context does not respect EMIT_PARSABLE_ERRORS

This patch makes Diagnostics.print_context respect EMIT_PARSABLE_ERRORS.

--- a/langkit/diagnostics.py
+++ b/langkit/diagnostics.py
@@ -178,18 +178,21 @@ def print_context(recovered=False):
     # Then we'll print the context we've kept
     last_file_info = ''
     for ctx_msg, ctx_loc in reversed(get_structured_context(recovered)):
-        # We only want to show the file information one time if it is the same
-        file_info = 'File "{}", '.format(col(ctx_loc.file, Colors.CYAN))
-        if last_file_info == file_info:
-            file_info = '  '
+        if EMIT_PARSABLE_ERRORS:
+            print "{}:{}: {}".format(ctx_loc.file, ctx_loc.line, ctx_msg)
         else:
-            last_file_info = file_info
-
-        print ('{file_info}line {line}, {msg}'.format(
-            file_info=file_info,
-            line=col(ctx_loc.line, Colors.CYAN),
-            msg=ctx_msg
-        ))
+            # We only want to show the file information one time if it is the same
+            file_info = 'File "{}", '.format(col(ctx_loc.file, Colors.CYAN))
+            if last_file_info == file_info:
+                file_info = '  '
+            else:
+                last_file_info = file_info
+                
+            print ('{file_info}line {line}, {msg}'.format(
+                file_info=file_info,
+                line=col(ctx_loc.line, Colors.CYAN),
+                msg=ctx_msg
+            ))
 
 
 def get_parsable_location():

Calling AST node "Length" breaks compilation

How to reproduce

Create an example project:

create-project.py repro

Change ExampleNode to Length:

$ sed -ie 's/ExampleNode/Length/' repro/language/parser.py
$ cd repro && python3 ./manage.py make
[...]
Compile
   [Ada]          parse.adb
parse.adb:137:44: "Length" is not visible
parse.adb:137:44: multiple use clauses cause hiding
parse.adb:137:44: hidden declaration at libreprolang-analysis.ads:88
parse.adb:137:44: hidden declaration at a-strunb.ads:92

   compilation of parse.adb failed

gprbuild: *** compilation phase failed
Build failed: error while running gprbuild -p -j8 -P/home/senier/tmp/langkit/length_issue/repro/build/src/mains.gpr -XBUILD_MODE=dev -XLIBRARY_TYPE=relocatable -XGPR_BUILD=relocatable -XXMLADA_BUILD=relocatable parse.adb:
    Command '['gprbuild', '-p', '-j8', '-P/home/senier/tmp/langkit/length_issue/repro/build/src/mains.gpr', '-XBUILD_MODE=dev', '-XLIBRARY_TYPE=relocatable', '-XGPR_BUILD=relocatable', '-XXMLADA_BUILD=relocatable', 'parse.adb']' returned non-zero exit status 4.

Expected behavior

Either reject Length as AST node name or (preferably) allow it's use in the parser by fully qualifying Ada.Strings.Unbounded.Length (maybe use clauses should be avoided in generated code alltogether).

Versions used

GNAT: Community 2020
Langkit: master (45df941)

Direct use of collections breaks with Python3.10

Various collections.* -> collections.abc.* aliases were removed in Python 3.10

This causes in libadalang tests like:

_Output:
_Traceback (most recent call last):
_  File "C:\libadalang\tmp\python__R918-040_rebindings\test.py", line 19, in <module>
_    ps = u.root.find(lambda n: n.text == 'Foo').parent
_  File "C:\libadalang\build\python\libadalang\__init__.py", line 2941, in find
_    return next(self.finditer(ast_type_or_pred, **kwargs))
_  File "C:\libadalang\build\python\libadalang\__init__.py", line 2972, in finditer
_    elif isinstance(ast_type_or_pred, collections.Sequence):
_AttributeError: module 'collections' has no attribute 'Sequence'

elif isinstance(ast_type_or_pred, collections.Sequence):

should be

        elif isinstance(ast_type_or_pred, collections.abc.Sequence):

(no change of import necessary, I believe)

setup.py not installing langkit.lexer

It seems to me that packages in setup.py should also include langkit.lexer.
Otherwise, the directory langkit/lexer is not copied when installing, and the compilation
of libadalang fails down the road...

Thanks !

Wrong build values for VSS dependancy.

I got these errors:

% python3 manage.py build-langkit-support --library-types=static,static-pic,relocatable --build-mode=prod
prettier_ada.gpr:16:04: value "static" is illegal for typed string "build"
vss_gnat.gpr:5:04: value "static" is illegal for typed string "build"
vss_text.gpr:7:04: value "static" is illegal for typed string "build"

How to recover?
[Edited]
Obviously, by selecting only relocatable:
% python3 manage.py build-langkit-support --library-types=relocatable --build-mode=prod
Indeed, this comes from the fact that by default VSS makefile builds only relocatable lib. build-all-lib shouldn't be the default as other tools?

Struggling with prefix directory

Obviously a noddy question, but not one I can seem to solve. I need to install libadalang so installing this is essential. My GNAT directory is in /opt as the community edition gives. Seems like whichever prefix value I choose I can't seem to get the libadalang installer to find it. Where it is supposed to be if I've got everything in /opt/GNAT?

Python version?

While running

python ada/manage.py generate -P

I got

 File "/Volumes/Miscellaneous2/libadalang/venv/src/langkit/langkit/documentation.py", line 27, in <module>
    from typing import (Any, Callable, Dict, List, Optional, Protocol, Set,
ImportError: cannot import name 'Protocol' from 'typing' (/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/typing.py)

Looks as though we need Python 3.8 now, not the 3.7+ noted in the README?

Cannot build langkit

With GCC-7.4.0 using these GNATColl commit ID's

$ cd langkit
$ export PYTHONPATH=$(pwd)
$ python2.7 scripts/build-langkit_support.py generate
$ python2.7 scripts/build-langkit_support.py --library-types relocatable build --build-mode=prod --gargs="-R"
Setup
   [mkdir]        object directory for project Langkit_Support
   [mkdir]        library directory for project Langkit_Support
Compile
   [Ada]          langkit_support-vectors.adb
   [Ada]          langkit_support-lexical_env.adb
   [Ada]          langkit_support-tree_traversal_iterator.adb
   [Ada]          langkit_support-token_data_handlers.adb
   [Ada]          langkit_support-symbols.adb
   [Ada]          langkit_support-adalog-unify_one_side.adb
   [Ada]          langkit_support-adalog-operations.adb
   [Ada]          langkit_support-adalog-abstract_relation.adb
   [Ada]          langkit_support.ads
   [Ada]          langkit_support-types.ads
   [Ada]          langkit_support-text.adb
   [Ada]          langkit_support-slocs.adb
   [Ada]          langkit_support-relative_get.adb
   [Ada]          langkit_support-packrat.adb
   [Ada]          langkit_support-iterators.adb
   [Ada]          langkit_support-images.adb
   [Ada]          langkit_support-hashes.adb
   [Ada]          langkit_support-diagnostics.adb
   [Ada]          langkit_support-cheap_sets.adb
   [Ada]          langkit_support-bump_ptr.adb
   [Ada]          langkit_support-bump_ptr-vectors.adb
   [Ada]          langkit_support-boxes.adb
   [Ada]          langkit_support-array_utils.adb
   [Ada]          langkit_support-adalog.ads
   [Ada]          langkit_support-adalog-unify_lr.adb
   [Ada]          langkit_support-adalog-unify.adb
   [Ada]          langkit_support-adalog-relations.adb
   [Ada]          langkit_support-adalog-pure_relations.adb
   [Ada]          langkit_support-adalog-predicates.adb
   [Ada]          langkit_support-adalog-main_support.adb
   [Ada]          langkit_support-adalog-logic_var.ads
   [Ada]          langkit_support-adalog-logic_ref.adb
   [Ada]          langkit_support-adalog-eq_same.adb
   [Ada]          langkit_support-adalog-debug.adb
langkit_support-adalog-eq_same.ads:126:14: implicit conversion of anonymous access value not allowed
langkit_support-adalog-eq_same.ads:126:14: implicit conversion of anonymous access value not allowed

   compilation of langkit_support-adalog-eq_same.adb failed
   compilation of langkit_support-adalog-main_support.adb failed

gprbuild: *** compilation phase failed
Build failed: error while running gprbuild -p -j8 -Pbuild/lib/gnat/langkit_support.gpr -XBUILD_MODE=prod -XLIBRARY_TYPE=relocatable -XGPR_BUILD=relocatable -XXMLADA_BUILD=relocatable -R:
    Command '['gprbuild', '-p', '-j8', '-Pbuild/lib/gnat/langkit_support.gpr', '-XBUILD_MODE=prod', '-XLIBRARY_TYPE=relocatable', '-XGPR_BUILD=relocatable', '-XXMLADA_BUILD=relocatable', '-R']' returned non-zero exit status 4

langkit is python 2.7 only?

Attempting to build with python 3.5+:

Traceback (most recent call last):
  File "ada/manage.py", line 14, in <module>
    from langkit.libmanage import ManageScript
  File "/construction/libadalang/libadalang-611f857/ada/../langkit/langkit/libmanage.py", line 17, in <module>
    from langkit.compile_context import Verbosity
  File "/construction/libadalang/libadalang-611f857/ada/../langkit/langkit/compile_context.py", line 1358
    key=lambda (s, n): n)
               ^
SyntaxError: invalid syntax

Python 2.7 is nearly EOL. In my opinion, no new development should target python 2.7.
At the very least, if this really is a restriction, it should be boldly proclaimed on the README as that is very important build information.

Empty diagnostics when using Python optimization

We have observed in RecordFlux that the diagnostics of our langkit-based parser is empty when python -O is used (AdaCore/RecordFlux#603). The root cause seems to be an assertion in the Python binding (librflxlang/__init__.py):

1120     @property
1121     def diagnostics(self):
1122         """
1123         Diagnostics for this unit.
1124         """
1125         count = _unit_diagnostic_count(self._c_value)
1126         result = []
1127         diag = Diagnostic._c_type()
1128         for i in range(count):
1129             assert _unit_diagnostic(self._c_value, i, ctypes.byref(diag))
1130             result.append(diag._wrap())
1131         return result

The function _unit_diagnostic seems to have side effects and is not executed when -O is used. If line 1129 is commented out, the issue also occurs without -O.

6313 _unit_diagnostic = _import_func(
6314     "rflx_unit_diagnostic",
6315     [AnalysisUnit._c_type, ctypes.c_uint, ctypes.POINTER(Diagnostic._c_type)],
6316     ctypes.c_int,
6317 )

We use -O for performance reasons, otherwise the assertions and contracts used in RecordFlux would make the execution significantly slower.

Versions

GNAT: Community 2020
Langkit: 21.0.0

Version control and test Langkit based Python parser

A few months ago, I wrote a Python parser leveraging Langkit.

Since we want to evolve toward langkit truly being a meta language framework with more than one language, I thought it would make sense to at least version control and run simple tests for the Python parser.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.