tree-sitter-grammars / tree-sitter-commonlisp Goto Github PK
View Code? Open in Web Editor NEWCommon Lisp grammar for tree-sitter
License: MIT License
Common Lisp grammar for tree-sitter
License: MIT License
tree-sitter --version
)tree-sitter 0.21.0 (1c55abb5308fe3891da545662e5df7ba28ade275)
Just started learning commonlisp, so there may be some subtleties here that I'm simply overlooking. However: Tree-sitter fails when encountering the square brackets in this piece of code:
(ql:quickload :cl-collider)
(in-package :sc-user)
(named-readtables:in-readtable :sc)
(setf *s* (make-external-server "localhost" :port 48800))
(server-boot *s*)
(defvar *synth*)
(setf *synth* (play(sin-osc.ar [320 321] 0 0.2)))
(free *synth*)
(server-quit *s*)
(server-query-all-nodes)
(source [0, 0] - [14, 0]
(list_lit [0, 0] - [0, 27]
value: (package_lit [0, 1] - [0, 13]
package: (sym_lit [0, 1] - [0, 3])
symbol: (sym_lit [0, 4] - [0, 13]))
value: (kwd_lit [0, 14] - [0, 26]
(kwd_symbol [0, 15] - [0, 26])))
(list_lit [1, 0] - [1, 21]
value: (sym_lit [1, 1] - [1, 11])
value: (kwd_lit [1, 12] - [1, 20]
(kwd_symbol [1, 13] - [1, 20])))
(list_lit [2, 0] - [2, 35]
value: (package_lit [2, 1] - [2, 30]
package: (sym_lit [2, 1] - [2, 17])
symbol: (sym_lit [2, 18] - [2, 30]))
value: (kwd_lit [2, 31] - [2, 34]
(kwd_symbol [2, 32] - [2, 34])))
(list_lit [4, 0] - [4, 57]
value: (sym_lit [4, 1] - [4, 5])
value: (sym_lit [4, 6] - [4, 9])
value: (list_lit [4, 10] - [4, 56]
value: (sym_lit [4, 11] - [4, 31])
value: (str_lit [4, 32] - [4, 43])
value: (kwd_lit [4, 44] - [4, 49]
(kwd_symbol [4, 45] - [4, 49]))
value: (num_lit [4, 50] - [4, 55])))
(list_lit [5, 0] - [5, 17]
value: (sym_lit [5, 1] - [5, 12])
value: (sym_lit [5, 13] - [5, 16]))
(list_lit [7, 0] - [7, 16]
value: (sym_lit [7, 1] - [7, 7])
value: (sym_lit [7, 8] - [7, 15]))
(ERROR [8, 0] - [14, 0]
value: (sym_lit [8, 1] - [8, 5])
value: (sym_lit [8, 6] - [8, 13])
value: (sym_lit [8, 15] - [8, 19])
value: (sym_lit [8, 20] - [8, 30])))
test.lisp 0.10 ms 2822 bytes/ms (ERROR [8, 0] - [14, 0])
I expect treesitter to be able to parse this normally, as the code compiles fine on sbcl/arch linux.
No response
My config:
require'nvim-treesitter.configs'.setup {
ensure_installed = {
"javascript", "typescript", "glimmer", "tsx",
"html", "jsdoc", "regex", "bash",
"toml", "html", "json",
"css", "lua", "commonlisp"
},
highlight = {
enable = true,
}
}
The tree-sitter tags
command can be used to implement "go to definition" functions. (In particular at github, I assume.)
It is documented here: https://tree-sitter.github.io/tree-sitter/code-navigation-systems
It does not work for tree-sitter-commonlisp. Right now (after fixing #12) if I run tree-sitter tags ~/prj/cl+ssl/cl-plus-ssl/src/package.lisp
I get just
No tags config found for path "/home/anton/prj/cl+ssl/cl-plus-ssl/src/streams.lisp"
To support tagging, the languiage impl should provide a queries/tags.scm file.
An example: The tree-sitter-ruby
After adding #14, the same tags
command as above prints function definition locations. The function names are not package qualified, but I think it may be even better that way for github code navigation currently, as symbol references produced by tree-sitter parse
command are also not package qualified (unless in the source code they have a package prefix)
initialize-instance | function def (50, 11) - (50, 30) `(defmethod initialize-instance :after ((stream ssl-stream)`
print-object | function def (61, 11) - (61, 23) `(defmethod print-object ((object ssl-stream) stream)`
stream-element-type | function def (73, 11) - (73, 30) `(defmethod stream-element-type ((stream ssl-stream))`
close | function def (76, 11) - (76, 16) `(defmethod close ((stream ssl-stream) &key abort)`
open-stream-p | function def (95, 11) - (95, 24) `(defmethod open-stream-p ((stream ssl-stream))`
stream-listen | function def (98, 11) - (98, 24) `(defmethod stream-listen ((stream ssl-stream))`
stream-read-byte | function def (109, 11) - (109, 27) `(defmethod stream-read-byte ((stream ssl-stream))`
stream-read-sequence | function def (123, 11) - (123, 31) `(defmethod stream-read-sequence ((stream ssl-stream) seq start end &key)`
stream-write-byte | function def (146, 11) - (146, 28) `(defmethod stream-write-byte ((stream ssl-stream) b)`
while | function def (154, 10) - (154, 15) `(defmacro while (cond &body body)`
stream-write-sequence | function def (157, 11) - (157, 32) `(defmethod stream-write-sequence ((stream ssl-stream) seq start end &key)`
stream-finish-output | function def (175, 11) - (175, 31) `(defmethod stream-finish-output ((stream ssl-stream))`
stream-force-output | function def (178, 11) - (178, 30) `(defmethod stream-force-output ((stream ssl-stream))`
install-nonblock-flag | function def (190, 7) - (190, 28) `(defun install-nonblock-flag (fd)`
install-nonblock-flag | function def (201, 7) - (201, 28) `(defun install-nonblock-flag (fd)`
install-nonblock-flag | function def (208, 7) - (208, 28) `(defun install-nonblock-flag (fd)`
install-nonblock-flag | function def (212, 7) - (212, 28) `(defun install-nonblock-flag (fd)`
install-handle-and-bio | function def (227, 7) - (227, 29) `(defun install-handle-and-bio (stream handle socket unwrap-stream-p)`
install-key-and-cert | function def (263, 7) - (263, 27) `(defun install-key-and-cert (handle key certificate)`
x509-certificate-names | function def (276, 7) - (276, 29) `(defun x509-certificate-names (x509-certificate)`
ssl-stream-handle | function def (289, 11) - (289, 28) `(defmethod ssl-stream-handle ((stream flexi-streams:flexi-stream))`
ssl-stream-x509-certificate | function def (292, 7) - (292, 34) `(defun ssl-stream-x509-certificate (ssl-stream)`
ssl-load-global-verify-locations | function def (295, 7) - (295, 39) `(defun ssl-load-global-verify-locations (&rest pathnames)`
ssl-set-global-default-verify-paths | function def (309, 7) - (309, 42) `(defun ssl-set-global-default-verify-paths ()`
ssl-check-verify-p | function def (316, 7) - (316, 25) `(defun ssl-check-verify-p ()`
ssl-verify-init | function def (330, 7) - (330, 22) `(defun ssl-verify-init (&key`
maybe-verify-client-stream | function def (348, 7) - (348, 33) `(defun maybe-verify-client-stream (ssl-stream verify-mode hostname)`
handle-external-format | function def (373, 7) - (373, 29) `(defun handle-external-format (stream ef)`
with-new-ssl | function def (378, 10) - (378, 22) `(defmacro with-new-ssl ((var) &body body)`
make-alpn-proto-string | function def (403, 7) - (403, 29) `(defun make-alpn-proto-string (protocols)`
make-ssl-client-stream | function def (412, 7) - (412, 29) `(defun make-ssl-client-stream (socket`
make-ssl-server-stream | function def (554, 7) - (554, 29) `(defun make-ssl-server-stream (socket`
get-selected-alpn-protocol | function def (599, 7) - (599, 33) `(defun get-selected-alpn-protocol (ssl-stream)`
stream-fd | function def (609, 12) - (609, 21) `(defgeneric stream-fd (stream)`
stream-fd | function def (614, 11) - (614, 20) `(defmethod stream-fd (stream) stream)`
stream-fd | function def (617, 11) - (617, 20) `(defmethod stream-fd ((stream sb-sys:fd-stream))`
stream-fd | function def (621, 11) - (621, 20) `(defmethod stream-fd ((stream system:fd-stream))`
stream-fd | function def (625, 11) - (625, 20) `(defmethod stream-fd ((stream ccl::basic-stream))`
stream-fd | function def (629, 11) - (629, 20) `(defmethod stream-fd ((stream stream))`
stream-fd | function def (634, 11) - (634, 20) `(defmethod stream-fd ((stream two-way-stream))`
stream-fd | function def (638, 11) - (638, 20) `(defmethod stream-fd ((stream stream))`
stream-fd | function def (642, 11) - (642, 20) `(defmethod stream-fd ((stream comm::socket-stream))`
stream-fd | function def (657, 13) - (657, 22) `(defmethod stream-fd ((stream system::socket-stream))`
Hi,
First off, thanks for making this available, it's very much appreciated.
As the title says, I think I found a minor bug where including a "@" in my function name breaks highlighting:
;; this highlights properly
(defun foo (bar)
(baz))
;; this doesn't and breaks all highlighting below
(defun @foo (bar)
(baz))
I'm on NVIM v0.5.1 with latest available treesitter:
-- Post-install/update hook with neovim command
use {
'nvim-treesitter/nvim-treesitter',
run = ':TSUpdate'
}
[...]
require'nvim-treesitter.configs'.setup {
ensure_installed = 'maintained',
highlight = { enable = true },
indent = { enable = true },
rainbow = {
enable = true,
extended_mode = false,
max_file_lines = nil,
}
}
Please let me know if this is not the right place to open an issue or if more info is needed.
For example:
(loop for s in symbols
maximizing (length (symbol-name s)))
Produces:
{Node source (0, 0) - (4, 0)}
{Node ERROR (1, 0) - (4, 0)}
{Node ( (1, 0) - (1, 1)} "("
{Node loop (1, 1) - (1, 5)} "loop"
{Node for (1, 6) - (1, 9)} "for"
{Node sym_lit (1, 10) - (1, 11)} "s"
{Node for_clause_word (1, 12) - (1, 14)}
{Node in (1, 12) - (1, 14)} "in"
{Node sym_lit (1, 15) - (1, 22)} "symbols"
I'm opening this issue as a heads up that we're planning to merge a change to the tree-sitter-clojure master branch.
IIUC, tree-sitter-commonlisp depends on a fork of tree-sitter-clojure at this commit, so if that's right, perhaps our upcoming intended change won't have any affect. I didn't analyze in detail how tree-sitter-clojure is used by tree-sitter-commonlisp, but for reference our intended change is here: sogaiu/tree-sitter-clojure#31
A brief overview of the changes follows.
kwd_lit
and sym_lit
will now contain up to 3 new nodes:
(kwd_lit
ns: (kwd_ns)
delimiter: "/"
name: (kwd_name))
(sym_lit
ns: (sym_ns)
delimiter: "/"
name: (sym_name))
Only the (kwd_name)
and (sym_name)
are required.
sym_lit
nodes can still contain metadata nodes (this is unchanged).
IIUC, tree-sitter-commonlisp redefines kwd_lit
and sym_lit
anyway so may be it wouldn't have had any effect on that account as well. We did, however, change some underlying bits that those two depended on though and didn't ascertain whether tree-sitter-commonlisp used any of them.
If we're mistaken that there would be no effect on tree-sitter-commonlisp from our upcoming changes, please let us know.
Thanks!
P.S. Our current plan is to merge around 2023-01-05 which is in about 2 weeks time.
On a more general note...
Though at this time we don't anticipate many changes going forward, we're planning to post certain change-related announcements to a pinned issue ahead of merging such changes.
If you'd like to keep track of what changes might be coming up, please have a look at the issue and consider subscribing :)
I'd love to include tree-sitter-commonlisp in difftastic, but this grammar is GPL unlike the other tree-sitter grammars available.
Would you be willing to relicense to something like MIT?
I understand this is a very personal thing, so no worries if you'd rather not.
@theHamsta , I am discussing with GitHub developers to use tree-sitter-commonlisp tagging for GitHub code navigation (https://github.com/orgs/community/discussions/55704).
I pointed them to my fork of tree-sitter-commonlisp, because I want to improve the tagging queries little by little. And if I will need to wait for you to merge every change, and wait for GitHub developers who also can not respond immediately, it will be very difficult for me to coordinate this, fix problems, etc.
Maybe you can give me write access to your repo? I will then point GitHub developers to your repo.
I will only use write access for tagging work. If I ever want to touch anything other than tagging, which is unlikely, I will open a pull request for you to review and not commit it by myself.
I've been testing this grammar against the .lisp
files in Sly, and I noticed that the grammar treats the following character as two literals:
#\Replacement_Character
This produces:
(source (char_lit) (sym_lit))
The grammar currently considers this to be an error:
(defpackage foo-bar
(:use cl))
It's not clear to me which grammar rules are preventing this.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.