juliadebug / juliainterpreter.jl Goto Github PK

View Code? Open in Web Editor NEW

159.0 159.0 33.0 2.67 MB

Interpreter for Julia code

License: Other

Julia 100.00%

juliainterpreter.jl's People

Contributors

Stargazers

Watchers

juliainterpreter.jl's Issues

RFC: Allow access to the interpreter state from interpreted code

It could be convenient in some situations to be able to access the interpreter state (most relevant is the current frame) from the code that itself is being interpreted.

This could be done (modulo macro hygiene) by something like

is_interpreted() = false
current_frame() = nothing

macro interpreted(ex)
    return :(
        if is_interpreted()
            frame = current_frame()
            $ex
        else
             nothing
        end
    )
end

One step of creating the Frame would then be to replace is_interpreted() with true and inject a reference to itself instead of current_frame().

One could then do things like

function f(x)
    print("hello")
    @interpreted print("I am being interpreted, pc currently at $(frame.pc)")
end

which could aid in debugging the interpreter itself but might also allow other cool stuff.

There are some considerations:

Are the use cases strong enough to implement it. I am thinking that e.g. file:line breakpoint support (without Revise) would be possible with this feature fairly trivially with a @breakpoint macro that inserts a breakpoint into the current frame.
With this you can in theory do things that are generally not allowed in julia due to not being optimizable, like local evals. I think that we don't really want people to use the interpreter to enable running non-standard julia code, but it is perhaps unlikely that people will use the interpreter for this.

Stacktraces point to the interpreter, not the code

Using this function from the test suite, we get this:

julia> f_exc_outer1()
ERROR: inner
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] f_exc_inner() at ./REPL[3]:2
 [3] f_exc_outer1() at ./REPL[6]:3
 [4] top-level scope at none:0

Unfortunately, when running in the interpreter

julia> @interpret f_exc_outer1()
ERROR: inner
Stacktrace:
 [1] evaluate_call!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:177
 [2] eval_rhs(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:292
 [3] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:399
 [4] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:437
 [5] finish!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:463
 [6] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:482 (repeats 3 times)
 [7] top-level scope at /home/tim/.julia/dev/JuliaInterpreter/src/JuliaInterpreter.jl:873

Since all the lines of the stacktrace point to the interpreter itself, it's quite difficult to tell that the interpreter is functioning "perfectly" and that the error is in the code being interpreted. Ideally we'd throw the same stacktrace in both cases.

Unfortunately, at the moment I'm not quite sure how to do that. We could craft a special InterpretedException type that records the correct stacktrace and then set up custom printing (or just use CapturedException). But then tests like

@test_throws ExcType @interpret(foo(args...; kwargs...))

will return the wrong exception type. What we really seem to need is the ability to craft a "fake" stacktrace that can be returned by catch_backtrace() (which is largely a ccall to jl_get_backtrace). Unfortunately the task excstack data are not accessible from Julia unless we do some scary pointer manipulations.

In #17 I "documented" this by adding a broken test, but this merits an issue of its own.

Fail with hcat and vcat

julia> @enter [1; 2]
typeof(f) = Int64
1
Tuple{Int64}
ERROR: no method found for the specified argument types
Stacktrace:
 [1] which(::Any, ::Any) at ./reflection.jl:823
 [2] #determine_method_for_expr#23(::Bool, ::Function, ::Expr) at /home/kx/git/ASTInterpreter2.jl/src/ASTInterpreter2.jl:321
 [3] (::ASTInterpreter2.#kw##determine_method_for_expr)(::Array{Any,1}, ::ASTInterpreter2.#determine_method_for_expr, ::Expr) at ./<missing>:0
 [4] #enter_call_expr#26(::Bool, ::Function, ::Expr) at /home/kx/git/ASTInterpreter2.jl/src/ASTInterpreter2.jl:406
 [5] enter_call_expr(::Expr) at /home/kx/git/ASTInterpreter2.jl/src/ASTInterpreter2.jl:406

julia> @enter [1 2]
typeof(f) = Int64
1
Tuple{Int64}
ERROR: no method found for the specified argument types
Stacktrace:
 [1] which(::Any, ::Any) at ./reflection.jl:823
 [2] #determine_method_for_expr#23(::Bool, ::Function, ::Expr) at /home/kx/git/ASTInterpreter2.jl/src/ASTInterpreter2.jl:321
 [3] (::ASTInterpreter2.#kw##determine_method_for_expr)(::Array{Any,1}, ::ASTInterpreter2.#determine_method_for_expr, ::Expr) at ./<missing>:0
 [4] #enter_call_expr#26(::Bool, ::Function, ::Expr) at /home/kx/git/ASTInterpreter2.jl/src/ASTInterpreter2.jl:406
 [5] enter_call_expr(::Expr) at /home/kx/git/ASTInterpreter2.jl/src/ASTInterpreter2.jl:406

Don't have this issue with cc05800. I guess it's introduced yesterday. Thanks.

MWE (sort of) for vecelement

The failure in vecelement is in the last test (figures). It results in an "this intrinsic must be compiled to be called" error. I think the problem is that a llvmcall is buried inside of a generated function, which defeats the compiled methods recognition?

using JuliaInterpreter
include("utils.jl")

module VecTest

Vec{N,T} = NTuple{N,Base.VecElement{T}}

# The following test mimic SIMD.jl
const _llvmtypes = Dict{DataType, String}(
    Float64 => "double",
    Float32 => "float",
    Int32 => "i32",
    Int64 => "i64"
)

@generated function vecadd(x::Vec{N, T}, y::Vec{N, T}) where {N, T}
    llvmT = _llvmtypes[T]
    func = T <: AbstractFloat ? "fadd" : "add"
    exp = """
    %3 = $(func) <$(N) x $(llvmT)> %0, %1
    ret <$(N) x $(llvmT)> %3
    """
    return quote
        Base.@_inline_meta
        Core.getfield(Base, :llvmcall)($exp, Vec{$N, $T}, Tuple{Vec{$N, $T}, Vec{$N, $T}}, x, y)
    end
end

end
m = VecTest


ex = quote
    a = (VecElement{Float64}(1.0), VecElement{Float64}(2.0))
    b = vecadd(a, a)
end


nstmts = 1000000

modexs, _ = JuliaInterpreter.split_expressions(m, ex)
stack = JuliaStackFrame[]
for modex in modexs
    frame = JuliaInterpreter.prepare_thunk(modex)
    nstmtsleft = nstmts
    while true
        ret, nstmtsleft = evaluate_limited!(stack, frame, nstmtsleft)
        break
        if isa(ret, Aborted)
            run_compiled(frame)
            break
        elseif isa(ret, Some)
            break
        end
    end
end

Crash in numbers test

julia> @interpret 2646693125139304345//842468587426513207 < pi
Unreachable reached at 0x7f58cd99668e

signal (4): Illegal instruction
in expression starting at no file:0
top-level scope at ./none:0
jl_fptr_trampoline at /usr/local/julia/julia-1.1/src/gf.c:1864
jl_toplevel_eval_flex at /usr/local/julia/julia-1.1/src/toplevel.c:758
jl_toplevel_eval_in at /usr/local/julia/julia-1.1/src/toplevel.c:793
eval at ./boot.jl:328 [inlined]
evaluate_foreigncall! at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:161
...

MWE for broken ```using```

Current build seems to have broken the using command.

using JuliaInterpreter
include("utils.jl")

module m
    using Test
end

ex = quote
    using Random
end

frame = JuliaInterpreter.prepare_thunk(m, ex)
JuliaInterpreter.finish_and_return!(frame)

This causes

macd@mlap:~/jlang/dev/JuliaInterpreter.jl/test$ julia randtest.jl
ERROR: Error while loading expression starting at /home/macd/jlang/dev/JuliaInterpreter.jl/test/randtest.jl:14
caused by [exception 1]
invalid lookup expr using Random
Stacktrace:
 [1] error(::String, ::Expr) at ./error.jl:42
 [2] eval_rhs(::Any, ::Frame, ::Expr) at /home/macd/jlang/dev/JuliaInterpreter.jl/src/interpret.jl:21
 [3] step_expr!(::Any, ::Frame, ::Any, ::Bool) at /home/macd/jlang/dev/JuliaInterpreter.jl/src/interpret.jl:478
 [4] step_expr! at /home/macd/jlang/dev/JuliaInterpreter.jl/src/interpret.jl:517 [inlined]
 [5] finish!(::Any, ::Frame, ::Bool) at /home/macd/jlang/dev/JuliaInterpreter.jl/src/commands.jl:14
 [6] finish_and_return! at /home/macd/jlang/dev/JuliaInterpreter.jl/src/commands.jl:29 [inlined]
 [7] finish_and_return! at /home/macd/jlang/dev/JuliaInterpreter.jl/src/commands.jl:33 [inlined] (repeats 2 times)
 [8] top-level scope at /home/macd/jlang/dev/JuliaInterpreter.jl/test/randtest.jl:14
 [9] include at ./boot.jl:325 [inlined]
 [10] include_relative(::Module, ::String) at ./loading.jl:1042
 [11] include(::Module, ::String) at ./Base.jl:29
 [12] exec_options(::Base.JLOptions) at ./client.jl:295
 [13] _start() at ./client.jl:464

PCRE :foreigncall error

julia> @interpret joinpath("/home/tim/src/julia-1/base", "sysimg.jl")
ERROR: invalid lookup expr ($(QuoteNode(tuple)))(:pcre2_match_8, $(QuoteNode("libpcre2-8")))
Stacktrace:
 [1] error(::String, ::Expr) at ./error.jl:42
 [2] lookup_expr at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:21 [inlined]
 [3] collect_args(::JuliaStackFrame, ::Expr) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:56
 [4] evaluate_foreigncall!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:126
 [5] eval_rhs(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:275
 [6] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:313
 [7] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:407
 [8] finish!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:433
 [9] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:449 (repeats 3 times)
 [10] evaluate_call!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:171
 [11] eval_rhs(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:273
 [12] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:380
 [13] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:407
 [14] finish!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:433
 [15] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:449 (repeats 3 times)
 [16] evaluate_call!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:171
 [17] eval_rhs(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:273
 [18] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:380
 [19] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:407
 [20] finish!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:433
 [21] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:449 (repeats 3 times)
 [22] evaluate_call!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:171
 [23] eval_rhs(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:273
 [24] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:380
 [25] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:407
 [26] finish!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:433
 [27] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:449 (repeats 3 times)
 [28] top-level scope at /home/tim/.julia/dev/JuliaInterpreter/src/JuliaInterpreter.jl:863

MWE failure in Libdl

using JuliaInterpreter
include("utils.jl")

module Libdl_test
    using Test
    using Libdl
end

m = Libdl_test

ex = quote
    dlls = Libdl.dllist()
end

frame = JuliaInterpreter.prepare_thunk(m, ex)
JuliaInterpreter.finish_and_return!(JuliaStackFrame[], frame)

Error evaluating some functions on expressions: "Symbol are not callable"

Working through a bit of the docs I encountered:

> JuliaInterpreter.determine_method_for_expr(:(sum([1,2])))
ERROR: MethodError: objects of type Symbol are not callable
Stacktrace:
 [1] #prepare_call#17(::Bool, ::Function, ::Any, ::Array{Any,1}) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:296
 [2] (::getfield(JuliaInterpreter, Symbol("#kw##prepare_call")))(::NamedTuple{(:enter_generated,),Tuple{Bool}}, ::typeof(JuliaInterpreter.prepare_call), ::Symbol, ::Array{Any,1}) at .\none:0
 [3] #determine_method_for_expr#18(::Bool, ::Function, ::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:385
 [4] determine_method_for_expr(::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:371
 [5] top-level scope at none:0

This also happens with for example

> frame = JuliaInterpreter.enter_call_expr(:(1+1))
ERROR: MethodError: objects of type Symbol are not callable

next_line! doesn't really seem to work properly

julia> function f(x)
           x = 2
           x = 3
           x = 4
       end
f (generic function with 1 method)

julia> frame = JuliaInterpreter.enter_call(f, 1);

julia> stack = [frame];

julia> JuliaInterpreter.linenumber(frame)
2

julia> JuliaInterpreter.next_line!(stack, frame)
JuliaProgramCounter(5)

julia> JuliaInterpreter.linenumber(frame) # Why are we not on line 3 now?
4

MWE of char crash

This reproduces the crash in the char tests.

using JuliaInterpreter
ex = quote
    a = ['0']
    b = ['a']
    c = [a; b]
end
frame = JuliaInterpreter.prepare_toplevel(Main, ex)
JuliaInterpreter.finish_and_return!(JuliaStackFrame[], frame, true)

with the result

Unreachable reached at 0x7f9c2cce8ab4

signal (4): Illegal instruction
[...]

Compiled resumers

#41 introduced the idea of using Compiled mode to finish tests (or preparatory steps) that take too long, so that an Abort doesn't nix later tests depending on the results. However, the implementation has several deficiencies:

it starts again from the beginning of the highest-level statement; instead, it should start at the lowest level on the stack and work its way back up. That will prevent it from repeating work (which is what it does now).
in some cases it may not fix the performance problem: it still has to interpret the frame that threw the Abort. Compiled means it won't need to recurse anymore, but if that frame happens to have an insanely long loop you'll still be stuck for a long time.

What we need instead is the notion of a compiled "resumer" function: starting from the method body, build a (temporary) function that schematically looks like this (would be built from lowered expressions in reality):

function #foo_resumer#34(slotvalues...)
    $(slotname[1]) = slotvalues[1]
...
    $(slotname[n]) = slotvalues[n]
    goto pc_where_it_quit
    $body
end

Then this could be called, in which case it would finish the computations in a truly compiled mode.

It has not escaped my attention that together with evaluate_limited!, this implements a possible means of addressing Julia's compiler-latency problems. Once this last piece is in place, I believe it would be a complete implementation of "interpret by default, then fall back gracefully to compiled mode for 'hot' portions of the code."

MWE failure in SHA

Currently the tests for the stdlib SHA fail in the interpreter and that causes a latent bug in the runtests.jl of SHA to come up all X's in the interpreter tests. The problem is that the function warn is used there, which is undefined, instead of the macro call @warn. I have submitted a PR for that, and once that is merged, then there are two failures in SHA, both related to sha1. The following code, when run from the JuliaInterpreter/test directory shows the problem. Interestingly, if nstmts is set to only 10000 instead of 1000000, then the test passes.

using JuliaInterpreter
include("utils.jl")

module SHA_test
    using SHA
    using Test

    sha1_result = "34aa973cd4c4daa4f61eeb2bdbad27316534016f"
    so_many_as_array = repeat([0x61], 1000000)
end

m = SHA_test

ex = quote
    ctx = SHA.SHA1_CTX()
    SHA.update!(ctx, so_many_as_array[1:2*SHA.blocklen(typeof(ctx))])
    SHA.update!(ctx, so_many_as_array[2*SHA.blocklen(typeof(ctx))+1:end])
    hash = bytes2hex(SHA.digest!(ctx))
    @test hash == sha1_result
end

function runex(ex; m=Main, nstmts=10000)
    modexs, _ = JuliaInterpreter.split_expressions(m, ex)
    stack = JuliaStackFrame[]
    for modex in modexs
        frame = JuliaInterpreter.prepare_thunk(modex)
        nstmtsleft = nstmts
        while true
            ret, nstmtsleft = evaluate_limited!(stack, frame, nstmtsleft)
            if isa(ret, Aborted)
                run_compiled(frame)
                break
            elseif isa(ret, Some)
                break
            end
        end
    end
end

runex(ex, m=m, nstmts=1000000)

Doctests are failing

Some doctests are failing but we are showing a lot of internals in the output for e.g. `JuliaStackFrame´ so it is perhaps not worth fixing these until something like #34 has been merged.

Also, the automatic doctest fixer in Documenter fails to fix these so I should probably figure out why.

Julia tests don't pass

Here's a status list for progress with JuliaInterpreter, running Julia's own test suite. It's organized by the number of passes, failures (ideally should be 0), errors (ideally 0), broken (these are not JuliaInterpreter's problem), and aborted blocks (tests that took too long, given the settings). Some tests error outside a @test (marked by "X" in the table below) and others cause Julia itself to exit (marked by ☠️)
The tests below were run on a multiprocessor server from the Linux command line with

$ JULIA_CPU_THREADS=8 julia --startup-file=no juliatests.jl --nstmts 1000000 --skip compiler

The --nstmts 1000000 allows you to control the maximum number of interpreter statements per lowered block of code; tests that require more than this are marked as being "aborted." The default setting is 10000 (10^4). The higher you make this number, in general the more tests that should finish, but of course also the longer the suite will take to run. On my laptop, running with 2 worker processes the entire suite takes less than 5 minutes to complete using the default settings.

The remaining arguments are the same as given to Julia's own test/runtests.jl: you can either provide a list of tests you want to run (e.g., julia --startup-file=no juliatests.jl ambiguous), or you can list some to skip (here, all the compiler/* tests). "Blank" indicates that one is running all the tests, so the line above runs everything except those in compiler/*.

The key point of having a status list is that it allows us to discover issues with JuliaInterpreter; consequently, the next step is to use these results to fix those problems. Help is very much wanted! Here are good ways to help out:

(moderate) investigate failures and file an issue with a MWE. Highest priority should probably go to ones that caused errors or process exit ~~(note: with the possible exceptions of channels, worlds, and arrayops, it appears that most such errors are due to a single cause, #28; deleting this block and rebuilding Julia fixes them)~~ (EDIT: all of these appear to be fixed now). Then would be error that occurs outside of tests (the Xs), errors that occur inside a @test (those marked as Errors by the test suite), failures, and of lowest priority the aborted blocks. Note that aborted blocks can lead to test failures due to repeating work (see #44), so many of these may go away if you increase nstmts. However, note that aborted blocks could indicate that the interpreter has incorrectly gotten itself stuck in an infinite loop (yes, the author has seen that happen), and as a consequence it's possible that some of these too are actually errors.
(hard) fix the bugs.

A good way to get started is to pick one test that's exhibiting problems, and uncomment these lines. Then, the easiest way to dive into this is to run tests in a REPL session, e.g.,

include("utils.jl")
const juliadir = dirname(dirname(Sys.BINDIR))
const testdir = joinpath(juliadir, "test")
configure_test()
nstmts = 10000
run_test_by_eval("ambiguous", joinpath(testdir, "ambiguous.jl"), nstmts)  # replace ambiguous with whatever test you want to run

from within JuliaInterpreter's test/ directory. If you get failures, make sure you first check whether they go away if you increase nstmts (typically by 10x or more).

When you see test errors, the expression printed right above it is the one causing the problem. Go into the source code and copy the relevant source lines into a quote block. Once you have a minimal expression ex that triggers a problem, do this:

modexs, _ = JuliaInterpreter.split_expressions(m, ex)
for modex in modexs
    frame = JuliaInterpreter.prepare_thunk(modex)
    nstmtsleft = nstmts
    while true
        ret, nstmtsleft = evaluate_limited!(frame, nstmtsleft, true)
        if isa(ret, Aborted)
            run_compiled(frame)
            break
        elseif isa(ret, Some)
            break
        end
    end
end

where m is the module you want to execute this in. You may want to do

module JuliaTests
using Test, Random
end
m = JuliaTests

to isolate the tests from your current session.

To diagnose problems in greater detail, uncommenting these lines can be a great first start.

Without further ado, here's the current list (note the time of the run to determine how current this is):

Julia Version 1.1.1-pre.0
Commit a84cf6f56c (2019-01-22 04:33 UTC)
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-6.0.1 (ORCJIT, haswell)
Test run at: 2019-02-26T11:36:49.456

Maximum number of statements per lowered expression: 1000000

Test file	Passes	Fails	Errors	Broken	Aborted blocks
ambiguous	62	0	0	2	0
subarray	281	0	0	0	1
strings/basic	87293	0	0	0	3
strings/search	522	0	0	0	0
strings/util	449	0	0	0	0
strings/io	12749	0	0	0	1
strings/types	2302688	0	0	0	3
unicode/utf8	19	0	0	0	0
core	X	X	X	X	X
worlds	X	X	X	X	X
keywordargs	126	0	1	0	0
numbers	1387242	0	0	0	9
subtype	X	X	X	X	X
char	1522	0	0	0	0
triplequote	28	0	0	0	0
intrinsics	44	0	0	0	0
dict	X	X	X	X	X
hashing	X	X	X	X	X
iobuffer	200	0	0	0	0
staged	55	5	0	0	0
offsetarray	341	11	0	0	1
arrayops	1833	0	0	0	7
tuple	483	0	1	0	0
reduce	292	0	0	0	2
reducedim	689	0	0	0	1
abstractarray	1791	0	0	0	1
intfuncs	4410	0	0	0	0
simdloop	X	X	X	X	X
vecelement	X	X	X	X	X
rational	97522	0	0	0	2
bitarray	897826	0	0	0	9
copy	511	0	1	0	1
math	X	X	X	X	X
fastmath	907	3	3	0	0
functional	95	0	0	0	0
iterators	1555	0	0	0	2
operators	12922	0	0	0	1
path	274	0	0	12	2
ccall	X	X	X	X	X
parse	10303	0	0	0	1
loading	2272	289	4	0	9
bigint	2156	0	0	0	4
sorting	4864	0	0	0	4
spawn	X	X	X	X	X
backtrace	5	9	12	1	0
exceptions	27	19	6	0	0
file	X	X	X	X	X
read	X	X	X	X	X
version	2468	0	0	0	1
namedtuple	152	0	8	1	0
mpfr	932	0	0	0	0
broadcast	418	0	5	0	2
complex	8250	0	0	2	1
floatapprox	49	0	0	0	0
reflection	X	X	X	X	X
regex	29	0	0	0	0
float16	124	0	0	0	0
combinatorics	98	0	0	0	1
sysinfo	2	0	0	0	0
env	53	0	0	0	0
rounding	112720	0	0	0	2
ranges	12109069	2	0	327755	7
mod2pi	80	0	0	0	0
euler	12	0	0	0	5
show	X	X	X	X	X
errorshow	X	X	X	X	X
sets	773	0	0	1	1
goto	X	X	X	X	X
llvmcall	X	X	X	X	X
llvmcall2	6	0	0	0	0
grisu	683	1	0	0	1
some	64	0	0	0	0
meta	X	X	X	X	X
stacktraces	X	X	X	X	X
docs	X	X	X	X	X
misc	X	X	X	X	X
threads	X	X	X	X	X
enums	88	0	0	0	0
cmdlineargs	X	X	X	X	X
int	10727	0	0	0	0
checked	1219	0	0	0	0
bitset	192	0	0	0	0
floatfuncs	134	0	0	0	1
boundscheck	X	X	X	X	X
error	30	0	0	0	0
cartesian	7	0	0	0	0
osutils	42	0	0	0	0
channels	X	X	X	X	X
iostream	6	0	2	0	0
secretbuffer	16	0	0	0	0
specificity	X	X	X	X	X
reinterpretarray	118	0	0	0	1
syntax	X	X	X	X	X
logging	117	2	0	0	0
missing	406	0	0	1	1
asyncmap	292	0	0	0	0
SparseArrays/higherorderfns	7000	79	0	73	7
SparseArrays/sparse	2184	0	0	0	19
SparseArrays/sparsevector	9921	0	0	0	5
Pkg/resolve	182	0	0	0	3
LinearAlgebra/triangular	33194	0	0	0	2
LinearAlgebra/qr	3120	0	0	0	1
LinearAlgebra/dense	7720	0	0	0	7
LinearAlgebra/matmul	711	0	0	0	3
LinearAlgebra/schur	390	0	0	0	1
LinearAlgebra/special	1068	0	0	0	3
LinearAlgebra/eigen	406	0	0	0	2
LinearAlgebra/bunchkaufman	5145	0	0	0	1
LinearAlgebra/svd	412	0	0	0	1
LinearAlgebra/lapack	778	2	0	0	3
LinearAlgebra/tridiag	1222	0	0	0	2
LinearAlgebra/bidiag	1946	0	0	0	1
LinearAlgebra/diagonal	1607	0	0	0	2
LinearAlgebra/cholesky	2194	0	0	0	1
LinearAlgebra/lu	1191	0	0	0	3
LinearAlgebra/symmetric	1982	0	0	0	1
LinearAlgebra/generic	430	0	0	0	3
LinearAlgebra/uniformscaling	338	0	0	0	0
LinearAlgebra/lq	1253	0	0	0	1
LinearAlgebra/hessenberg	40	0	0	0	0
LinearAlgebra/blas	628	0	0	0	1
LinearAlgebra/adjtrans	253	0	0	0	1
LinearAlgebra/pinv	288	0	0	0	1
LinearAlgebra/givens	1840	0	0	0	1
LinearAlgebra/structuredbroadcast	408	0	0	0	2
LibGit2/libgit2	219	2	59	1	0
Dates/accessors	X	X	X	X	X
Dates/adjusters	X	X	X	X	X
Dates/query	988	0	0	0	0
Dates/periods	681	0	0	0	0
Dates/ranges	349123	0	0	0	5
Dates/rounding	296	0	0	0	0
Dates/types	171	0	0	0	0
Dates/io	258	0	0	0	1
Dates/arithmetic	318	0	0	0	0
Dates/conversions	160	0	0	0	0
Base64	1015	0	0	0	1
CRC32c	658	0	6	0	0
DelimitedFiles	80	0	1	0	1
FileWatching	X	X	X	X	X
Future	0	0	0	0	0
InteractiveUtils	104	3	2	0	4
Libdl	X	X	X	X	X
Logging	35	1	0	0	1
Markdown	232	0	0	0	0
Mmap	131	0	0	0	1
Printf	701	38	0	0	0
Profile	10	0	0	0	2
REPL	990	0	0	5	0
Random	203081	4	0	0	7
SHA	X	X	X	X	X
Serialization	105	1	1	0	0
Sockets	X	X	X	X	X
Statistics	606	0	0	0	4
SuiteSparse	770	0	0	0	0
Test	X	X	X	X	X
UUIDs	22	0	0	0	0
Unicode	752	0	0	0	0

Hashing test fails

Copied from #13 (comment), CC @macd

Ran the hashing module following instructions above (using today's master) most of the tests pass but saw the following:

mod = Main.JuliaTests
ex = quote
#= /home/macd/julia/test/hashing.jl:188 =#
let a = Expr(:block, Core.TypedSlot(1, Any)), b = Expr(:block, Core.TypedSlot(1, Any)), c = Expr(:block, Core.TypedSlot(3, Any))
#= /home/macd/julia/test/hashing.jl:191 =#
#= /home/macd/julia/test/hashing.jl:191 =# @test a == b && hash(a) == hash(b)
#= /home/macd/julia/test/hashing.jl:192 =#
#= /home/macd/julia/test/hashing.jl:192 =# @test a != c && hash(a) != hash(c)
#= /home/macd/julia/test/hashing.jl:193 =#
#= /home/macd/julia/test/hashing.jl:193 =# @test b != c && hash(b) != hash(c)
end
end
hashing: Error During Test at /home/macd/julia/test/hashing.jl:191
Test threw exception
Expression: a == b && hash(a) == hash(b)
syntax: Slot objects should not occur in an AST
hashing: Error During Test at /home/macd/julia/test/hashing.jl:192
Test threw exception
Expression: a != c && hash(a) != hash(c)
syntax: Slot objects should not occur in an AST
hashing: Error During Test at /home/macd/julia/test/hashing.jl:193
Test threw exception
Expression: b != c && hash(b) != hash(c)
syntax: Slot objects should not occur in an AST

Optimizing getfields should perhaps be a bit less aggressive.

Replacing getfields with their value before evaluating the frame can sometimes be dangerous. Consider the two functions

function f(x)
      if false
          return y
      else
          return x
      end
  end

function g()
    eval(:(z = 2))
    return z
end

Running these in a new julia session, they can both be evaluated without problems:

> f(1)
1

> g()
2

However, trying to interpret them:

> @interpret f(1)
ERROR: UndefVarError: y not defined
Stacktrace:
 [1] lookup_global_refs!(::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:764
 [2] optimize!(::Core.CodeInfo, ::Module) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:797
 [3] #JuliaFrameCode#2(::Bool, ::Bool, ::Bool, ::Bool, ::Type, ::Method, ::Core.CodeInfo) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:106

> @interpret g()
ERROR: UndefVarError: z not defined
Stacktrace:
 [1] lookup_global_refs!(::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:764
 [2] optimize!(::Core.CodeInfo, ::Module) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:797
 [3] #JuliaFrameCode#2(::Bool, ::Bool, ::Bool, ::Bool, ::Type, ::Method, ::Core.CodeInfo) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:106

In fact, we can get wrong results:

q = 0

function h()
    eval(:(q = 2))
    return q
end

Compiled mode:

> h()
2

interpreted mode:

> @interpret h()
0

Perhaps we should only do this optimization when the value is defined and const (or a method).

Slot not assigned

This failure prevents successful execution of the dict tests:

julia> A = [1]
1-element Array{Int64,1}:
 1

julia> wkd = WeakKeyDict()
WeakKeyDict{Any,Any} with 0 entries

julia> @interpret setindex!(wkd, 2, A)
ERROR: slot _2 with name x not assigned
Stacktrace:
 [1] error(::String, ::Core.SlotNumber, ::String, ::Symbol, ::String) at ./error.jl:42
 [2] lookup_var(::JuliaStackFrame, ::Core.SlotNumber) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:13
 [3] #collect_args#5(::Bool, ::Function, ::JuliaStackFrame, ::Expr) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:56
 [4] collect_args at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:131 [inlined]
 [5] #evaluate_call!#8 at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:167 [inlined]
 [6] evaluate_call! at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:165 [inlined]
 [7] eval_rhs(::Compiled, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:315
 [8] _step_expr!(::Compiled, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:439
...
 [42] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:464
 [43] finish!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:507
 [44] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:526 (repeats 3 times)
 [45] top-level scope at /home/tim/.julia/dev/JuliaInterpreter/src/JuliaInterpreter.jl:1063

Chopped off docstring for whereis

JuliaInterpreter.jl/src/interpret.jl

Lines 701 to 703 in d5fbe82

 When `frame` represents top-level code, 

 """ 

 function CodeTracking.whereis(framecode::JuliaFrameCode, pc)

Maybe there should be a continuation here? :)

Another :foreigncall issue

julia> using JuliaInterpreter

julia> ex = quote
           if sizeof(JLOptions) === ccall(:jl_sizeof_jl_options, Int, ())
           else
               ccall(:jl_throw, Cvoid, (Any,), "Option structure mismatch")
           end
       end;

julia> frame = JuliaInterpreter.prepare_toplevel(Base, ex)
JuliaStackFrame(JuliaInterpreter.JuliaFrameCode(Base, CodeInfo(
1 ─ %1 = sizeof(JLOptions)
│   %2 = $(Expr(:foreigncall, :(:jl_sizeof_jl_options), :Int, :(($(QuoteNode(Core.svec)))()), :(:ccall), 0))
│   %3 = %1 === %2
└──      goto #3 if not %3
2 ─      return
3 ─ %6 = $(Expr(:foreigncall, :(:jl_throw), :Cvoid, :(($(QuoteNode(Core.svec)))(Any)), :(:ccall), 1, "Option structure mismatch"))
└──      return %6
), Core.TypeMapEntry[#undef, #undef, #undef, #undef, #undef, #undef, #undef], BitSet([1, 2, 3, 6]), false, false, true), Union{Nothing, Some{Any}}[], Any[#undef, #undef, #undef, #undef, #undef, #undef, #undef], Any[], Int64[], Base.RefValue{Any}(nothing), Base.RefValue{JuliaInterpreter.JuliaProgramCounter}(JuliaProgramCounter(1)), Dict{Symbol,Int64}(), Any[])

julia> JuliaInterpreter.finish_and_return!(JuliaStackFrame[], frame, true)
ERROR: invalid lookup expr ($(QuoteNode(Core.svec)))()
Stacktrace:
 [1] error(::String, ::Expr) at ./error.jl:42
 [2] lookup_expr at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:21 [inlined]
 [3] #collect_args#5(::Bool, ::Function, ::JuliaStackFrame, ::Expr) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:56
 [4] #collect_args at ./none:0 [inlined]
 [5] evaluate_foreigncall!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:137
 [6] eval_rhs(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:286
 [7] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:388
 [8] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:418
 [9] finish!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:444
 [10] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:460
 [11] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:465
 [12] top-level scope at none:0

Latency on first call unacceptable

This package loads quite quickly:

julia> @time(using JuliaInterpreter)
  0.146759 seconds (178.45 k allocations: 11.704 MiB, 3.25% gc time)

but the first usage is slow:

julia> @time @interpret 1+1
  3.875351 seconds (1.34 M allocations: 65.670 MiB, 1.16% gc time)
2

Especially if Revise is going to be transitioning to using JuliaInterpreter for signature-extraction, this needs improvement.

Error in copy_codeinfo `no method matching copy(::Type{Any})` in Debugger tests on nightly

https://travis-ci.org/JuliaDebug/Debugger.jl/jobs/494959238#L468-L494

Reorganize the source

Some possible improvements (perhaps to be made at a time after all merge-worthy PRs are in):

centralize is* methods?
move optimize!-related code to separate file?
consolidate macro-related code?
organize the various prepare_* methods more logically?

getproperty in struct definition crashes interpreter

> JuliaInterpreter.interpret!(JuliaInterpreter.JuliaStackFrame[], Main, Meta.parse(
  """
  module Toplevel
  module DatesMode
      abstract type Period end
  end
  
  struct Beat <: DatesMode.Period
      value::Int64
  end
  end
  """
  , 1)[1])

ERROR: unknown call f getproperty
Stacktrace:
 [1] error(::String, ::Function) at .\error.jl:42
 [2] lookup_or_eval(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:96
 [3] evaluate_structtype!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:247
 [4] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:373
 [5] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at
C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:436
 [6] finish!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:479
 [7] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:498
 [8] finish_and_return! at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:503 [inlined]
 [9] interpret!(::Array{JuliaStackFrame,1}, ::Module, ::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:783
 [10] interpret!(::Array{JuliaStackFrame,1}, ::Module, ::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:768
 [11] top-level scope at none:0

Subtype check is not handled by the @interpret macro

julia> @interpret Float32 <: Int32
ERROR: MethodError: no method matching Float32(::Type{Int32})
Closest candidates are:
  Float32(::Int8) at float.jl:60
  Float32(::Int16) at float.jl:60
  Float32(::Int32) at float.jl:60
  ...
Stacktrace:
 [1] #prepare_call#45(::Bool, ::Function, ::Any, ::Array{Any,1}) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:306
 [2] (::getfield(JuliaInterpreter, Symbol("#kw##prepare_call")))(::NamedTuple{(:enter_generated,),Tuple{Bool}}, ::typeof(JuliaInterpreter.prepare_call), ::Type, ::Array{Any,1}) at .\none:0
 [3] #determine_method_for_expr#22(::Bool, ::Function, ::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:385
 [4] #determine_method_for_expr at .\none:0 [inlined]
 [5] #enter_call_expr#25(::Bool, ::Function, ::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:649
 [6] enter_call_expr(::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:649
 [7] top-level scope at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:886

Keyword arguments in @interpret are not properly evaluated

> using Gadfly

> @interpret plot(y=[1,2,3])
ERROR: MethodError: no method matching evalmapping(::Nothing, ::Expr)
Closest candidates are:
  evalmapping(::Any, ::AbstractArray) at C:\Users\Kristoffer\.julia\packages\Gadfly\09PWZ\src\mapping.jl:185
  evalmapping(::Any, ::Function) at C:\Users\Kristoffer\.julia\packages\Gadfly\09PWZ\src\mapping.jl:186
  evalmapping(::Any, ::Distributions.Distribution) at C:\Users\Kristoffer\.julia\packages\Gadfly\09PWZ\src\mapping.jl:187
Stacktrace:
 [1] evalmapping!(::Dict{Symbol,Any}, ::Nothing, ::Gadfly.Data) at C:\Users\Kristoffer\.julia\packages\Gadfly\09PWZ\src\mapping.jl:220
 [2] plot at C:\Users\Kristoffer\.julia\packages\Gadfly\09PWZ\src\Gadfly.jl:327 [inlined]
 [3] #plot#66(::Base.Iterators.Pairs{Symbol,Expr,Tuple{Symbol},NamedTuple{(:y,),Tuple{Expr}}}, ::Function) at C:\Users\Kristoffer\.julia\packages\Gadfly\09PWZ\src\Gadfly.jl:308
 [4] maybe_evaluate_builtin(::JuliaStackFrame, ::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\builtins-julia1.1.jl:42
 [5] #evaluate_call!#9(::typeof(JuliaInterpreter.finish_and_return!), ::Function, ::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:191
 [6] evaluate_call!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:182
 [7] eval_rhs(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:327
 [8] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:452
 [9] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter,
::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:478
 [10] finish!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:530
 [11] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:558 (repeats 3 times)
 [12] top-level scope at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:1128

Removing the @interpret and it works.

MWE of tuple test failure

This looks more isolated to JuliaInterpreter.

using JuliaInterpreter
ex = quote
    struct BitPerm_19352
        p::NTuple{8,UInt8}
        function BitPerm(p::NTuple{8,UInt8})
            new(p)
        end
        BitPerm_19352(xs::Vararg{Any,8}) = BitPerm(map(UInt8, xs))
    end
end
frame = JuliaInterpreter.prepare_toplevel(Main, ex)
JuliaInterpreter.finish_and_return!(JuliaStackFrame[], frame, true)

gives

ERROR: TypeError: in typeassert, expected Expr, got Array{Any,1}
Stacktrace:
 [1] eval_rhs(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/gunnar/.julia/dev/JuliaInterpreter/src/interpret.jl:307
 [2] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/gunnar/.julia/dev/JuliaInterpreter/src/interpret.jl:408
 [3] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/gunnar/.julia/dev/JuliaInterpreter/src/interpret.jl:436
 [4] finish!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/gunnar/.julia/dev/JuliaInterpreter/src/interpret.jl:479
 [5] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/gunnar/.julia/dev/JuliaInterpreter/src/interpret.jl:498
 [6] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Bool) at /home/gunnar/.julia/dev/JuliaInterpreter/src/interpret.jl:503
 [7] top-level scope at REPL[4]:1

Suspicious error throwing for CodeInfo

This error call in

JuliaInterpreter.jl/src/interpret.jl

Lines 169 to 172 in 8ee5829

 if isa(f, CodeInfo) 

 error("CodeInfo") 

 ret = finish_and_return!(Compiled(), enter_call_expr(frame, call_expr)) 

 else

stood out to me as a bit suspicious when reading the code.

Should this just be an @assert !(f isa CodeInfo) and the call to finish_and_return! removed?

Failure in `lookup_global_refs!` for a method of `convert`

I doubt this is an issue with breakpoint specifically, just that I happened to discover it that way:

julia> breakpoint(convert)
ERROR: UndefVarError: c not defined
Stacktrace:
 [1] lookup_global_refs!(::Expr) at /home/tim/.julia/dev/JuliaInterpreter/src/JuliaInterpreter.jl:685
 [2] optimize!(::Core.CodeInfo, ::Module) at /home/tim/.julia/dev/JuliaInterpreter/src/JuliaInterpreter.jl:718
 [3] #JuliaFrameCode#2(::Bool, ::Bool, ::Bool, ::Bool, ::Type, ::Method, ::Core.CodeInfo) at /home/tim/.julia/dev/JuliaInterpreter/src/JuliaInterpreter.jl:79
 [4] Type at /home/tim/.julia/dev/JuliaInterpreter/src/breakpoints.jl:0 [inlined]
 [5] breakpoint(::Method, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/breakpoints.jl:167
 [6] breakpoint(::Function, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/breakpoints.jl:181
 [7] breakpoint(::Function) at /home/tim/.julia/dev/JuliaInterpreter/src/breakpoints.jl:180
 [8] top-level scope at none:0

Rename and/or split out prepare_toplevel?

I'm mulling over a split in Revise:

ReviseCore handles the diff/patch functionallity ("Revise's core mission"). No dependency on juliainterpreter. Sets up the data for CodeTracking.pkgfiles. Intended for people who want their Julia sessions to start up fast without a dependency on JuliaInterpreter.
Revise (which will depend on ReviseCore) will add the signature-parsing and thus handle everything that depends on it: the location updates (CodeTracking.whereis, correcting line numbers in stack traces) and source management (CodeTracking.definition). In the long run, signature-parsing will leverage JuliaInterpreter.

It looks like ReviseCore needs JuliaInterpreter.prepare_toplevel. This function is probably misnamed now: the modern functionality is better-described as split_expressions or something. (It takes an expression and breaks it into chunks, organized by module-of-evaluation.) Historically, this function also lowered each expression and built a JuliaStackFrame, but then I discovered that you can't lower expression n+1 until you have evaled expression n. (What if expression n defines a new macro required for expression n+1?)

Thoughts about how to share this code among two packages that don't need to talk to one another? It's not that much code, maybe best is to code-copy as it seems silly to have a package just for one function. But copying has its problems too.

MWE for failures in file

The first two errors are due to throwing the wrong exception. The third error has my new favorite error message: sigatomic_end called in non-sigatomic region

using JuliaInterpreter
include("utils.jl")

module file_test
    using Test
end

m = file_test

ex = quote
    @test_throws ArgumentError download("good", "ba\0d") # Thrown: UndefVarError
    @test_throws ArgumentError download("ba\0d", "good") # Thrown: UndefVarError
    n = tempname()
    w = open(n, "a")
    write(w, "A")
    flush(w)  #  sigatomic_end called in non-sigatomic region
    #close(w)
    #rm(n)
end

frame = JuliaInterpreter.prepare_thunk(m, ex)
JuliaInterpreter.finish_and_return!(JuliaStackFrame[], frame)

More breakpoint automatic modes

Currently break_on_error[] can take values true or false; if true, it breaks on any uncaught exception. Some other things that might be cool:

break on caught exceptions (can help figure out tricky try/catch-induced issues)
break on test failure
break on warn

I haven't investigated the feasibility of these, so I have no idea what implementation would look like.

Caching of builtins.jl might be problematic

Two different versions of julia using the same version of ASTInterpreter2 will use the same package folder. However, since the julia versions are different, they might require different builtins.jl. Perhaps the builtins.jl file should be namespaced based on the julia version.

Error on `which` for field

julia> @interpret(Vararg.body.body.name)
typeof(f) = DataType
Vararg{T,N}
Tuple{Symbol}
ERROR: TypeError: in Type, in parameter, expected Type, got Vararg{T,N}
Stacktrace:
 [1] signature_type(::Any, ::Any) at ./reflection.jl:722
 [2] which(::Any, ::Any) at ./reflection.jl:999
 [3] #prepare_call#41(::Bool, ::Function, ::Type, ::Array{Any,1}) at /home/tim/.julia/dev/JuliaInterpreter/src/JuliaInterpreter.jl:275
 [4] (::getfield(JuliaInterpreter, Symbol("#kw##prepare_call")))(::NamedTuple{(:enter_generated,),Tuple{Bool}}, ::typeof(JuliaInterpreter.prepare_call), ::Type, ::Array{Any,1}) at ./none:0
 [5] #determine_method_for_expr#17(::Bool, ::Function, ::Expr) at /home/tim/.julia/dev/JuliaInterpreter/src/JuliaInterpreter.jl:362
 [6] #determine_method_for_expr at ./none:0 [inlined]
 [7] #enter_call_expr#20(::Bool, ::Function, ::Expr) at /home/tim/.julia/dev/JuliaInterpreter/src/JuliaInterpreter.jl:625
 [8] enter_call_expr(::Expr) at /home/tim/.julia/dev/JuliaInterpreter/src/JuliaInterpreter.jl:625
 [9] top-level scope at /home/tim/.julia/dev/JuliaInterpreter/src/JuliaInterpreter.jl:860

Turns out to be JuliaLang/julia#30995

Interpreting Core.Compiler functions: no method matching lastindex(::JuliaInterpreter.Compiled)

julia> ci = @code_lowered gcd(10, 20);

julia> cfg = Core.Compiler.compute_basic_blocks(ci.code)

> @interpret Core.Compiler.SNCA(cfg)
ERROR: MethodError: no method matching lastindex(::Compiled)
Closest candidates are:
  lastindex(::Markdown.MD) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Markdown\src\parse\parse.jl:26
  lastindex(::Base64.Buffer) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Base64\src\buffer.jl:19
  lastindex(::Cmd) at process.jl:940
  ...
Stacktrace:
 [1] #enter_call_expr#52(::Bool, ::Function, ::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:771
 [2] enter_call_expr(::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:769
 [3] top-level scope at C:\Users\Kristoffer\.julia\packages\JuliaInterpreter\JckC5\src\JuliaInterpreter.jl:973

Empty module crashes interpreter

julia> stack = JuliaInterpreter.JuliaStackFrame[];

julia> JuliaInterpreter.interpret!(stack, Main, :(module Foo end))
ERROR: UndefVarError: ret not defined
Stacktrace:
 [1] interpret!(::Array{JuliaStackFrame,1}, ::Module, ::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:788
 [2] interpret!(::Array{JuliaStackFrame,1}, ::Module, ::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\JuliaInterpreter.jl:769
 [3] top-level scope at none:0

MWE for failing test for errorshow.jl or "Varargs, the undead of the Julia Type system"

It turns out that the failure in errorshow.jl has to do with handling
Varargs. Due to JuliaLang/julia#30995, there is
a check in prepare_call in construct.jl that will punt if any of the args
is a Vararg. But that is the problem, there is not an easy way to test the
different forms of Vararg that come through.

The test is currently elseif any(x->isa(x, Type) && x <: Vararg, allargs)
however, one form of that comes through is x = Vararg{AbstractString,N} where {N}
and isa(x, Type) tests true but x <: Vararg is false. typeof(x) returns
a UnionALL. Another arg that comes through is Vararg{AbstractString,N}, that
is, there is no where clause?! I don't understand how that is possible.

I could not construct a test to get these two, and only these two, using type checking.
For the purposes of testing, I used the brute force approach of
elseif any(x->isa(x, Type) && occursin("Vararg", string(x)), allargs) The unit
tests for JuliaInterpreter run OK with this and the following MWE passes, but there should
be a better way to do that, so I did not prepare a PR.

using JuliaInterpreter
include("utils.jl")

module m
    method_c1(x::Float64, s::AbstractString...) = true
    buf = IOBuffer()
    me = Base.MethodError(method_c1,(1, 1, ""))
end

ex = quote
    Base.show_method_candidates(buf, me)
end

# TypeError: in Type, in parameter, expected Type, got Vararg{AbstractString,N} where N

frame = JuliaInterpreter.prepare_thunk(m, ex)
JuliaInterpreter.finish_and_return!(frame)

precompile fails to cache finish_and_return!

While working on #60 I noticed that despite the appropriate precompile calls, finish_and_return!(::Vector{JuliaStackFrame}, ::JuliaStackFrame) is still being inferred the first time it gets called. For @interpret 1+1 it's the only significant inference overhead, around 0.25s on the machine I tested. This is not a major priority, but it does merit investigation (possible precompile bug).

For reference, here's the patch I use for timing inference:

diff --git a/base/compiler/typeinfer.jl b/base/compiler/typeinfer.jl
index bbeea301fe..822875ccad 100644
--- a/base/compiler/typeinfer.jl
+++ b/base/compiler/typeinfer.jl
@@ -535,8 +535,17 @@ function typeinf_code(method::Method, @nospecialize(atypes), sparams::SimpleVect
     return (frame.src, widenconst(result.result))
 end
 
-# compute (and cache) an inferred AST and return type
+const inf_timing = []
 function typeinf_ext(linfo::MethodInstance, params::Params)
+    tstart = ccall(:jl_clock_now, Float64, ())
+    ret = _typeinf_ext(linfo, params)
+    tstop = ccall(:jl_clock_now, Float64, ())
+    push!(inf_timing, (tstart, linfo, tstop))
+    return ret
+end
+
+# compute (and cache) an inferred AST and return type
+function _typeinf_ext(linfo::MethodInstance, params::Params)
     method = linfo.def::Method
     for i = 1:2 # test-and-lock-and-test
         i == 2 && ccall(:jl_typeinf_begin, Cvoid, ())

I've found this really helps. SnoopCompile essentially times the wrong thing. (It times e.g., LLVM, but we don't cache native code yet so that's pretty irrelevant.)

MWE for InteractiveUtils fail

julia> f = x-> x > 1 ? "foo" : nothing

julia> iob = IOBuffer()
IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=0, maxsize=Inf, ptr=1, mark=-1)

julia> ioc = IOContext(iob, :color => true)

julia> @interpret code_warntype(ioc, f, Tuple{Int64})
ERROR: ArgumentError: input string is empty or only contains whitespace
Stacktrace:
 [1] maybe_evaluate_builtin(::JuliaStackFrame, ::Expr) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\builtins.jl:94
 [2] #evaluate_call!#9(::typeof(JuliaInterpreter.finish_and_return!), ::Function, ::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:176
 [3] evaluate_call!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:176
 [4] eval_rhs(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:302
 [5] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:411
 [6] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:436
 [7] finish!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:479
 [8] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:498 (repeats 2 times)
 [9] #evaluate_call!#9(::typeof(JuliaInterpreter.finish_and_return!), ::Function, ::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:498
 ... (the last 7 lines are repeated 7 more times)
 [59] evaluate_call!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:176
 [60] eval_rhs(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:302
 [61] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:411
 [62] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:436
 [63] finish!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:479
 [64] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at C:\Users\Kristoffer\Debugging\JuliaInterpreter\src\interpret.jl:498 (repeats 3 times)

Conditional breakpoints have a scope problem

In the expression that defines a breakpoint, the scope for the condition is module-toplevel. You can do cool stuff when all expressions are known at toplevel:

julia> using JuliaInterpreter
[ Info: Recompiling stale cache file /home/tim/.julia/compiled/v1.1/JuliaInterpreter/PliIn.ji for JuliaInterpreter [aa1ae85d-cabe-5617-a682-6adf51b2e16a]

julia> foo(x, y) = x+y
foo (generic function with 1 method)

julia> call_foo(x, y) = foo(x, y)
call_foo (generic function with 1 method)

julia> @breakpoint foo(1,1) y>x
breakpoint(foo(x, y) in Main at REPL[2]:1, 1)

julia> @interpret call_foo(1, 1)
2

julia> @interpret call_foo(1, 2)
breakpoint(foo(x, y) in Main at REPL[2]:1, 1)

julia> remove()

julia> @breakpoint foo(1,1) y>x^2
breakpoint(foo(x, y) in Main at REPL[2]:1, 1)

julia> @interpret call_foo(2, 4)
6

julia> @interpret call_foo(2, 5)
breakpoint(foo(x, y) in Main at REPL[2]:1, 1)

But things break if you reference objects that are not toplevel:

julia> remove()

julia> let
           localfunc(x) = x^2
           @breakpoint foo(1,1) y>localfunc(x)
           @interpret call_foo(2, 4)
       end
ERROR: UndefVarError: localfunc not defined
Stacktrace:
 [1] ##slotfunction#368(::JuliaStackFrame) at ./none:0
 [2] shouldbreak(::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/breakpoints.jl:62
 [3] #evaluate_call!#9(::typeof(JuliaInterpreter.finish_and_return!), ::Function, ::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/breakpoints.jl:58
 [4] evaluate_call!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:180
 [5] eval_rhs(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:320
 [6] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:445
 [7] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:471
 [8] finish!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:523
 [9] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:551 (repeats 3 times)
 [10] top-level scope at /home/tim/.julia/dev/JuliaInterpreter/src/JuliaInterpreter.jl:1122
 [11] top-level scope at REPL[12]:4

It's worth noting that Debugger's eval_code has the same problem:

julia> using Debugger

julia> stack = @make_stack call_foo(2,3)
1-element Array{JuliaStackFrame,1}:
 JuliaStackFrame(JuliaInterpreter.JuliaFrameCode(call_foo(x, y) in Main at REPL[3]:1, CodeInfo(
1 ─ %1 = ($(QuoteNode(foo)))(x, y)
└──      return %1
), Union{Compiled, TypeMapEntry}[TypeMapEntry(nothing, Tuple{typeof(foo),Int64,Int64}, #undef, #undef, 0, 0, MethodInstance for foo(::Int64, ::Int64), true, false, false), #undef], JuliaInterpreter.BreakpointState[#undef, #undef], BitSet([1]), false, false, true), Union{Nothing, Some{Any}}[Some(call_foo), Some(2), Some(3)], Any[#undef, #undef], Any[], Int64[], Base.RefValue{Any}(nothing), Base.RefValue{JuliaInterpreter.JuliaProgramCounter}(JuliaProgramCounter(1)), Dict(:y=>3,Symbol("#self#")=>1,:x=>2), Any[])

julia> state = Debugger.DebuggerState(stack, nothing)
Debugger.DebuggerState(JuliaStackFrame[JuliaStackFrame(JuliaFrameCode(call_foo(x, y) in Main at REPL[3]:1, CodeInfo(
1 ─ %1 = ($(QuoteNode(foo)))(x, y)
└──      return %1
), Union{Compiled, TypeMapEntry}[TypeMapEntry(nothing, Tuple{typeof(foo),Int64,Int64}, #undef, #undef, 0, 0, MethodInstance for foo(::Int64, ::Int64), true, false, false), #undef], JuliaInterpreter.BreakpointState[#undef, #undef], BitSet([1]), false, false, true), Union{Nothing, Some{Any}}[Some(call_foo), Some(2), Some(3)], Any[#undef, #undef], Any[], Int64[], RefValue{Any}(nothing), RefValue{JuliaProgramCounter}(JuliaProgramCounter(1)), Dict(:y=>3,Symbol("#self#")=>1,:x=>2), Any[])], 1, nothing, nothing, nothing, Base.RefValue{REPL.LineEdit.Prompt}(#undef), nothing, nothing)

julia> Debugger.eval_code(state, "x")
(true, 2)

julia> Debugger.eval_code(state, "x^2")
(true, 4)

julia> let
           localfunc(x) = x^2
           Debugger.eval_code(state, "localfunc(x)")
       end
(false, (UndefVarError(:localfunc), Union{Ptr{Nothing}, InterpreterIP}[Ptr{Nothing} @0x00007f0345b6b928, Ptr{Nothing} @0x00007f0345b3b0b4, Ptr{Nothing} @0x00007f0345b8d050, Ptr{Nothing} @0x00007f0345c8cc05, Ptr{Nothing} @0x00007f0345c8d0c6, Ptr{Nothing} @0x00007f0345c8cd98, Ptr{Nothing} @0x00007f0345c8dbb2, Ptr{Nothing} @0x00007f0345c8e173, InterpreterIP(CodeInfo(
1 ─      #self# = $(QuoteNode(call_foo))
│        x = $(QuoteNode(2))
│        y = $(QuoteNode(3))
│        374 = localfunc(x)
│   %5 = 374
│   %6 = (Core.tuple)(#self#, x, y)
│   %7 = (Core.tuple)(%5, %6)
└──      return %7
), 0x0000000000000003), Ptr{Nothing} @0x00007f0345c8ecbc  …  Ptr{Nothing} @0x00007f0345c8ecbc, Ptr{Nothing} @0x00007f0345b5607c, Ptr{Nothing} @0x00007f0345b56f34, Ptr{Nothing} @0x00007f03398edc82, Ptr{Nothing} @0x00007f0345b1e8b5, Ptr{Nothing} @0x00007f0339a8b230, Ptr{Nothing} @0x00007f0339a8b494, Ptr{Nothing} @0x00007f0345b1e8b5, Ptr{Nothing} @0x00007f0345b3b961, Ptr{Nothing} @0x0000000000000000]))

MWE of failure in test for show

The test of redirect_stdout fails because stdout doesn't really appear to be redirected. Note that this happens for redirect_stderr as well.

ex = quote
    rdout, wrout = redirect_stdout()
    @test wrout === stdout
end

`linenumber` function assumes all statements have location info

However, that's not necessarily the case. It is legal for the index to be 0 corresponding to unknown location info. E.g.

diff --git a/src/interpret.jl b/src/interpret.jl
index 1a22e66..f7c3830 100644
--- a/src/interpret.jl
+++ b/src/interpret.jl
@@ -616,6 +616,7 @@ isgotonode(node) = isa(node, GotoNode) || isexpr(node, :gotoifnot)
 linenumber(frame) = linenumber(frame, frame.pc[])
 function linenumber(frame, pc)
     codeloc = frame.code.code.codelocs[pc.next_stmt]
+    codeloc == 0 && return nothing
     return frame.code.scope isa Method ?
         frame.code.code.linetable[codeloc].line :
         codeloc

but obviously the callers would have to be adjusted to deal with that, so there may be a better way.

Invalid pointer

This causes X on the arrayops tests.

julia> using JuliaInterpreter

julia> TT = Union{UInt8, Int8}
Union{Int8, UInt8}

julia> a = TT[0x0, 0x1]
2-element Array{Union{Int8, UInt8},1}:
 0x00
 0x01

julia> pa = pointer(a)
Ptr{Union{Int8, UInt8}} @0x00007fbf5c864950

julia> @interpret unsafe_store!(pa, 0x1, 2)
ERROR: pointerset: invalid pointer
Stacktrace:
 [1] maybe_evaluate_builtin(::JuliaStackFrame, ::Expr) at /home/tim/.julia/dev/JuliaInterpreter/src/builtins-julia1.1.jl:121
 [2] #evaluate_call!#9(::typeof(JuliaInterpreter.finish_and_return!), ::Function, ::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:189
 [3] evaluate_call!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:180
 [4] eval_rhs(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Expr, ::JuliaInterpreter.JuliaProgramCounter) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:315
 [5] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:439
 [6] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:464
 [7] finish!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:507
 [8] finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::JuliaInterpreter.JuliaProgramCounter, ::Bool) at /home/tim/.julia/dev/JuliaInterpreter/src/interpret.jl:526 (repeats 3 times)
 [9] top-level scope at /home/tim/.julia/dev/JuliaInterpreter/src/JuliaInterpreter.jl:1063

Plan for tags + release

@timholy What are your thoughts about tagging + releasing and making a bit of noise so that people start trying things out. Do you feel the interpreter needs some more maturing based on the julia tests you are running or is it in a stable enough state for external use?

MWE of failure in Random

So rand! should be non allocating. This fails in the interpreter.

ex = quote
    A = [0.0, 0.0]
    @test @allocated(rand!(A)) == 0
end

Reproduction of hashing test failure

For me, hashing is an X rather than a ☠️. Anyway, this is as far as I can reduce it. Probably not minimal but rather brittle.

With /tmp/hashing_test.jl containing

vals = Any[
    [0], [1], [2],
    [2, 1, 2, 3], [2, 3, 2, 1], [2, 2, 3, 2],
    [0, 0], [0, 0, 0], [0, 1], [1, 0],
    [0, 0, 1], [0, 1, 0], [1, 0, 0], [0, 1, 2],
    [0, 0], [1, 0], [0, 1], [0, 0], [0, 0],
    [5, 1], [1, 1], [0, 2], [0, 2], [4, 0],
    [0, 0], [1, 0], [0, 0, 2], [0 0 7],
    [4 0 0], [0 2 4],
]

b = vals

then

include("utils.jl")
dotest1("hashing", "/tmp/hashing_test.jl", 10000)

results in

ERROR: UndefVarError: vals not defined
Stacktrace:
 [1] _step_expr!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Any, ::JuliaProgramCounter, ::Bool) at /home/gunnar/.julia/dev/JuliaInterpreter/src/interpret.jl:419
 [2] limited_finish_and_return!(::Array{JuliaStackFrame,1}, ::JuliaStackFrame, ::Int64, ::JuliaProgramCounter, ::Bool) at /home/gunnar/.julia/dev/JuliaInterpreter/test/utils.jl:197
 [3] (::getfield(Main, Symbol("##11#13")){Int64})(::JuliaStackFrame) at /home/gunnar/.julia/dev/JuliaInterpreter/test/utils.jl:210
 [4] #invokelatest#1 at ./essentials.jl:761 [inlined]
 [5] invokelatest at ./essentials.jl:760 [inlined]
 [6] lower!(::Any, ::Dict{Module,Array{Expr,1}}, ::Array{LineNumberNode,1}, ::Module, ::Expr) at /home/gunnar/.julia/dev/JuliaInterpreter/test/utils.jl:96
 [7] lower_incrementally!(::Any, ::Dict{Module,Array{Expr,1}}, ::Array{LineNumberNode,1}, ::Expr, ::Module, ::Expr) at /home/gunnar/.julia/dev/JuliaInterpreter/test/utils.jl:74
 [8] lower_incrementally!(::Any, ::Dict{Module,Array{Expr,1}}, ::Array{LineNumberNode,1}, ::Expr, ::Module, ::Array{Any,1}) at /home/gunnar/.julia/dev/JuliaInterpreter/test/utils.jl:83
 [9] lower_incrementally!(::Any, ::Dict{Module,Array{Expr,1}}, ::Array{LineNumberNode,1}, ::Expr, ::Module, ::Expr) at /home/gunnar/.julia/dev/JuliaInterpreter/test/utils.jl:47
 [10] lower_incrementally! at /home/gunnar/.julia/dev/JuliaInterpreter/test/utils.jl:40 [inlined]
 [11] lower_incrementally at /home/gunnar/.julia/dev/JuliaInterpreter/test/utils.jl:36 [inlined]
 [12] (::getfield(Main, Symbol("##10#12")){String,Int64,Module,Expr})() at /home/gunnar/.julia/dev/JuliaInterpreter/test/utils.jl:296
 [13] cd(::getfield(Main, Symbol("##10#12")){String,Int64,Module,Expr}, ::String) at ./file.jl:96
 [14] dotest1(::String, ::String, ::Int64) at /home/gunnar/.julia/dev/JuliaInterpreter/test/utils.jl:293
 [15] top-level scope at REPL[2]:1

Removing e.g. the first element from vals makes the test pass.

Renaming?

As we near release, perhaps we should think a bit more about naming. Here are a few potential gotchas, without yet proposing specific fixes:

frame.code.code.code is awkward
are JuliaFrameCode and JuliaStackFrame sufficient clear as to their purpose?
step_expr! vs _step_expr!
we have various prepare_* but then also build_frame
we have both isfoo and is_foo names

Stack is not empty after returning from top function with a try catch

julia> function f_exc_outer()
      try
          f_exc_inner()
      catch err
          return err
      end
  end

julia> function f_exc_inner()
      error()
  end

julia> stack = JuliaStackFrame[];

julia> frame = JuliaInterpreter.enter_call(f_exc_outer);

julia> JuliaInterpreter.finish_and_return!(stack, frame)
ErrorException("")

julia> length(stack)
2

It seems like we have some frames left in the stack even though we have exited from the top level function.

Use `Base.copy(::CodeInfo)` rather than custom copy_codeinfo

I was about to suggest the following diff to prevent the method at:

JuliaInterpreter.jl/src/JuliaInterpreter.jl

Line 550 in ca07ee9

function copy_codeinfo(code::CodeInfo)

diff --git a/src/JuliaInterpreter.jl b/src/JuliaInterpreter.jl
index 1caa2e0..f528b47 100644
--- a/src/JuliaInterpreter.jl
+++ b/src/JuliaInterpreter.jl
@@ -552,7 +552,7 @@ function copy_codeinfo(code::CodeInfo)
     for (i, name) in enumerate(fieldnames(CodeInfo))
         if isdefined(code, name)
             val = getfield(code, name)
-            ccall(:jl_set_nth_field, Cvoid, (Any, Csize_t, Any), newcode, i-1, val===nothing || isa(val, Type) ? val : copy(val))
+            ccall(:jl_set_nth_field, Cvoid, (Any, Csize_t, Any), newcode, i-1, val===nothing || isa(val, Union{Type, Method}) ? val : copy(val))
         end
     end
     return newcode

from trying to copy a Method, but then I remembered that base has an implementation of copy(::CodeInfo) that should just be used instead.

Markdown tests fail seemingly due to undefined variable in Base

Running the Markdown tests errors with UndefVarError(:ERROR_JIT_BADOPTION) and indeed, I cannot find such a variable anywhere in my Julia folder. It is likely supposed to be generated into pcre_h.jl but is not (at least on my system).

JuliaLang/julia#31081

	When `frame` represents top-level code,
	"""
	function CodeTracking.whereis(framecode::JuliaFrameCode, pc)

	if isa(f, CodeInfo)
	error("CodeInfo")
	ret = finish_and_return!(Compiled(), enter_call_expr(frame, call_expr))
	else

juliadebug / juliainterpreter.jl Goto Github PK

juliainterpreter.jl's People

Contributors

Stargazers

Watchers

Forkers

juliainterpreter.jl's Issues

Recommend Projects

Recommend Topics

Recommend Org