end-of-buffer about flycheck-julia HOT 29 CLOSED

using CUDArt

function rotation_matrix(phi)
    [cos(phi) -sin(phi);
     sin(phi)  cos(phi)]
end

r1 = rotation_matrix(0.1)
r2 = rotation_matrix(0.3)
r3 = rotation_matrix(0.5)
rr = r1 * r2 * r3

r

gives the error, without the using CUDArt line it works fine.

lintfile("test.jl") gives an error message before printing the errors:

INFO: Precompiling module CUDArt.
ERROR: LoadError: InitError: unknown error (CUDA error #999, ERROR_UNKNOWN)
 in macro expansion at /home/gkraemer/.julia/v0.5/CUDAdrv/src/base.jl:127 [inlined]
 in init(::Int64) at /home/gkraemer/.julia/v0.5/CUDAdrv/src/init.jl:10
 in __init__() at /home/gkraemer/.julia/v0.5/CUDAdrv/src/init.jl:27
 in _include_from_serialized(::String) at ./loading.jl:150
 in _require_from_serialized(::Int64, ::Symbol, ::String, ::Bool) at ./loading.jl:187
 in _require_search_from_serialized(::Int64, ::Symbol, ::String, ::Bool) at ./loading.jl:217
 in require(::Symbol) at ./loading.jl:371
 in include_from_node1(::String) at ./loading.jl:488
 in macro expansion; at ./none:2 [inlined]
 in anonymous at ./<missing>:?
 in eval(::Module, ::Any) at ./boot.jl:234
 in process_options(::Base.JLOptions) at ./client.jl:242
 in _start() at ./client.jl:321
during initialization of module CUDAdrv
while loading /home/gkraemer/.julia/v0.5/CUDArt/src/CUDArt.jl, in expression starting on line 5
/home/gkraemer/test2.jl:14 E321 r: use of undeclared symbol

from flycheck-julia.

also I notice that the linter gets very slow with the using CUDArt line.

from flycheck-julia.

(ert-deftest flycheck-julia-test-query ()
  :tags '(query)
  (sleep-for 5)
  (flycheck-julia-server-start)
  (should
   (with-temp-buffer
     (insert-string "\ny\n")
     (ignore-errors
       (flycheck-julia-server-query 'flycheck-julia)
       (flycheck-julia-server-query 'flycheck-julia)
       (flycheck-julia-server-query 'flycheck-julia)
       (flycheck-julia-server-query 'flycheck-julia)
       (flycheck-julia-server-query 'flycheck-julia)
       (flycheck-julia-server-query 'flycheck-julia)
       (flycheck-julia-server-query 'flycheck-julia)
       (flycheck-julia-server-query 'flycheck-julia)
       (flycheck-julia-server-query 'flycheck-julia)
       (flycheck-julia-server-query 'flycheck-julia)
       (flycheck-julia-server-query 'flycheck-julia)
       (flycheck-julia-server-query 'flycheck-julia))
     ;; Print out the contents of the julia server process buffer
     (sleep-for 5)
     (let ((oldbuf (current-buffer)))
       (set-buffer (get-buffer "*julia-linter*"))
       (message (buffer-substring-no-properties (point-min) (point-max)))
       (set-buffer oldbuf))

     ;; (sleep-for 5)
     (cl-search
      "undeclared symbol"
      (aref (nth 0 (flycheck-julia-server-query 'flycheck-julia)) 6)))

   )
  (flycheck-julia-server-stop)
  (sleep-for 5)
  (kill-buffer "*julia-linter*"))

the internal part (with-temp-buffer ...) works as expected from an emacs session with an already running lint server but the whole test fails.

from flycheck-julia.

I can't reproduce this issue.

julia> lintfile("test.jl")
INFO: dynamic import: CUDArt
WARNING: is_windows is deprecated, use Sys.iswindows instead.
Stacktrace:
 [1] depwarn(::String, ::Symbol) at ./deprecated.jl:70
 [2] is_windows() at ./deprecated.jl:31
 [3] lintpkgforfile(::String, ::Lint.LintContext) at /home/fengyang/.julia/v0.7/Lint/src/cli.jl:18
 [4] lintfile(::String, ::String) at /home/fengyang/.julia/v0.7/Lint/src/cli.jl:66
 [5] lintfile(::String) at /home/fengyang/.julia/v0.7/Lint/src/cli.jl:52
 [6] eval(::Module, ::Expr) at ./repl/REPL.jl:3
 [7] eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./repl/REPL.jl:67
 [8] macro expansion at ./repl/REPL.jl:98 [inlined]
 [9] (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:73
while loading no file, in expression starting on line 0
test.jl:13 E321 r: use of undeclared symbol
test.jl:1 W101 CUDArt: unfortunately, Lint could not determine the exports of this module

from flycheck-julia.

you also get a different error message. The general pattern I observe is that linting fails for about 20-30 seconds after startup, because it receives incomplete JSON objects from the server. My best guess is that errors and warnings somehow make other connections get flushed prematurely.

Linting the above file works now, I am not sure what changed, but the slowness issue with the using CUDArt line persists.

from flycheck-julia.

I am trying to reproduce it with a package that errors on loading:

module Dummy
error("HELP")
end # module Dummy

linting a file with using Dummy works fine and there is now slowness.

from flycheck-julia.

The error message above is an is_windows deprecation warning and I think unrelated. I will try my best to look into the slowness issue. Could you give a order of magnitude comparison -- is it much slower than linting a module that does not loaderror?

from flycheck-julia.

Scrap the above, after restarting the lint server I can reproduce the slowness with a file containing using Dummy

from flycheck-julia.

it has a noticeable delay, around half a second for every query to the server.

from flycheck-julia.

Right, the dynamic imports are always slow -- when linting a file containing using X, Lint needs to dynamically import X to get the list of symbols it exports. In the future, this will be improved by caching X's exported symbols upon importing it once, and then not requiring future imports for linting the same file again.

from flycheck-julia.

ok, if the slowness is cleared, there remain the incomplete JSON objects after starting the server, I think the Julia linter for atom has the same issue.

from flycheck-julia.

What's the issue with incomplete JSON objects... are those never completed, or is it just that there is temporarily nothing in the buffer before a JSON object is completed?

from flycheck-julia.

i am not sure, but for the first ~20-30 seconds Emacs receives either empty strings or incomplete JSON objects, e.g. "[{", it is usually the first part that arrives. Maybe this could be solved client side by waiting for the next part of the object. I should probably make some kind of debug mode.

from flycheck-julia.

Opened an issue for the debug mode: #14

from flycheck-julia.

That is really quite strange... How are you parsing the JSON? Shouldn't it be parsed only when the object is complete?

from flycheck-julia.

If that is too complex of logic to implement, we can try something like separate all the JSON responses with newlines.

from flycheck-julia.

simple: query -> wait for response -> process response -> if there is an error, ignore it.
Works quite well ~20-30 seconds after the server starts.

from flycheck-julia.

Right, my questions is what does "wait for response" mean in a TCP context. What delimits the end of the response?

from flycheck-julia.

I use the Emacs function (accept-process-output) which returns as soon as there is output (which is probably responsible for the incomplete JSON, if the server does not return everything in a single piece) or after a certain time (which propagates possible performance problems from the server). I seem to have the same problem as described here.

Is there a way to guarantee that the JSON comes back in one piece?

from flycheck-julia.

and: what combination of characters determines the end of the returned JSON object? "}]"?

from flycheck-julia.

I don't think there is a way to guarantee that it comes back in one piece, because I don't know if it's meaningful for anything sent over a socket. However we can do something like always print a newline between JSON responses (I'll need to check if we are not doing that already.)

from flycheck-julia.

A quick review seems to indicate that we aren't currently printing newlines: https://github.com/tonyhffong/Lint.jl/blob/master/src/server.jl#L101

We can do that and then the definition of the response being complete would be once a newline is received. Would this work for you?

from flycheck-julia.

this would just imply changing print to println? If there are no new lines inside the JSON, this would probably be the best way to do it. I hope this does not break compatibility with the current version, my unit tests currently don't test for this.

from flycheck-julia.

Now I make multiple requests and concatenate the output, to debug I print the iteration number and the string I got back from the server. After just starting the server it looked like below, after that, it takes ~8 concatenations to complete the object:

...
570
... empty string ...
571
[
...
585
[{
...
588
[{"location":{"file":
589
[{"location":{"file":"/home/gkraemer/test.jl","position":
590
[{"location":{"file":"/home/gkraemer/test.jl","position":[
591
[{"location":{"file":"/home/gkraemer/test.jl","position":[
592
... the complete object ...

from flycheck-julia.

It is not really possible to guarantee that you'll receive everything at the same time because of how sockets work. The best we can do is terminate things with a newline.

For what it's worth, the JSON.jl library can handle receiving JSON well, because it will continue to listen to input until the JSON object is completed. If this is possible with emacs lisp JSON parser then that would be ideal...

from flycheck-julia.

I think I have a fairly well working version now, I just introduced a lot of bugs elsewhere which I still have to work out. It works without the newline at the end (I am looking for "}]"), but it shouldn't be a problem to look for "\n" or "}]\n". In case you decide to add the newline at the end it would be fine with me, just give me a heads up, because it would break the current version.

For what it's worth, the JSON.jl library can handle receiving JSON well, because it will continue to listen to input until the JSON object is completed. If this is possible with emacs lisp JSON parser then that would be ideal...

I could try processing the output string each time I receive something from the server and simply keep concatenating while this fails, but I am not sure if this is the best solution.

from flycheck-julia.

I think the newline would be worthwhile. Expect it in Lint master soon (to be released in 0.6.0, probably a while off), but it will probably not be backported to the 0.5.x series. If there is a way to make the emacs linter work with or without the newline, that would be best.

from flycheck-julia.

It shouldn't be a problem to make it work with both.

from flycheck-julia.

@TotalVerb I did some changes and figured out that the best way to make sure, that everything from the server arrived is, to simply wait until the connection closes.

from flycheck-julia.