pycket / pycket Goto Github PK

View Code? Open in Web Editor NEW

254.0 17.0 25.0 55.17 MB

A rudimentary Racket implementation using RPython

License: MIT License

Makefile 0.30% Shell 0.35% Python 51.92% Racket 34.35% C 0.02% Scheme 13.06%

pycket racket pypy rpython

pycket's People

Contributors

Stargazers

Watchers

pycket's Issues

strange parsing error only when translated

Run the following program in a translated pycket:

#lang racket/base
(require racket/contract/base)

I get the following error:

RPython traceback:
  File "pycket_entry_point.c", line 822, in entry_point
  File "pycket_expand.c", line 432, in load_json_ast_rpython
  File "pycket_expand.c", line 2962, in _to_module
  File "pycket_expand.c", line 10573, in _to_ast
  File "pycket_expand.c", line 22392, in _to_require
  File "pycket_expand.c", line 28632, in expand_to_ast
  File "pycket_expand.c", line 2962, in _to_module
  File "pycket_expand.c", line 10573, in _to_ast
  File "pycket_expand.c", line 22392, in _to_require
  File "pycket_expand.c", line 28632, in expand_to_ast
  File "pycket_expand.c", line 2962, in _to_module
  File "pycket_expand.c", line 10573, in _to_ast
  File "pycket_expand.c", line 22392, in _to_require
  File "pycket_expand.c", line 28632, in expand_to_ast
  File "pycket_expand.c", line 2962, in _to_module
  File "pycket_expand.c", line 10573, in _to_ast
  File "pycket_expand.c", line 22392, in _to_require
  File "pycket_expand.c", line 28632, in expand_to_ast
  File "pycket_expand.c", line 2962, in _to_module
  File "pycket_expand.c", line 10573, in _to_ast
  File "pycket_expand.c", line 22392, in _to_require
  File "pycket_expand.c", line 28632, in expand_to_ast
  File "pycket_expand.c", line 2962, in _to_module
  File "pycket_expand.c", line 10573, in _to_ast
  File "pycket_expand.c", line 22392, in _to_require
  File "pycket_expand.c", line 28632, in expand_to_ast
  File "pycket_expand.c", line 2962, in _to_module
  File "pycket_expand.c", line 10573, in _to_ast
  File "pycket_expand.c", line 22392, in _to_require
  File "pycket_expand.c", line 28632, in expand_to_ast
  File "pycket_expand.c", line 2962, in _to_module
  File "pycket_expand.c", line 10573, in _to_ast
  File "pycket_expand.c", line 22392, in _to_require
  File "pycket_expand.c", line 28632, in expand_to_ast
  File "pycket_expand.c", line 2962, in _to_module
  File "pycket_expand.c", line 10573, in _to_ast
  File "pycket_expand.c", line 22392, in _to_require
  File "pycket_expand.c", line 28632, in expand_to_ast
  File "pycket_expand.c", line 2962, in _to_module
  File "pycket_expand.c", line 13424, in _to_ast
  File "pycket_expand.c", line 8436, in _to_ast
  File "pycket_expand.c", line 15770, in _to_ast
  File "pycket_expand.c", line 22808, in to_lambda
  File "pycket_expand.c", line 7413, in _to_ast
  File "pycket_expand.c", line 7413, in _to_ast
  File "pycket_expand.c", line 7413, in _to_ast
  File "pycket_expand.c", line 6804, in _to_ast
  File "pycket_expand.c", line 7077, in _to_ast
  File "pycket_expand.c", line 6662, in _to_ast
  File "pycket_expand.c", line 20718, in to_value
  File "pycket_expand.c", line 24342, in _to_num
  File "pycket_json.c", line 2059, in Visitor_visit_NUMBER
  File "rpython_rtyper_lltypesystem_rstr.c", line 2819, in ll_int__rpy_stringPtr_Signed
Fatal RPython error: ValueError
Aborted (core dumped)

But this doesn't happen when not translated.

Support unsafe operations

I don't know if this requires anything more than just not doing the class checks.

Fix string parsing in JSON

In fa55728 I changed this to something that allows a few more real racket files to parse, but is still wrong.

The broken racket file was generating JSON with:

{'string' : "\\\\"}

Implement exceptions

Things to do here:

Throw real exceptions on runtime-system failures
Allow handling exceptions

r_dict assertion results in failure of the assignment conversion pass

The offending line of code can be found here
https://github.com/samth/pycket/blob/master/pycket/interpreter.py#L534

This assertion always fails in the compiled Pycket interpreter, while interpreting Pycket still works fine.

Handle literal hash tables

Like #hash() which can appear in Racket source.

Traces starting at tiny functions

So it seems a common problem pycket has is that it starts to trace from tiny functions that don't do very much and definitely are not loops. We would like it to stop doing that. One approach is the the cont_ast branch that seems to over-specialize.

make racket's psyntax benchmark work

Benchmark is here, see all the files that start with psyntax: https://github.com/plt/racket/tree/master/pkgs/racket-pkgs/racket-benchmarks/tests/racket/benchmarks/common

Needs various racket-y things to be able to work unmodified.

Implement rationals

Right now we implement / as what racket calls quotient. This is (a) wrong and (b) faster than implementing exact rationals.

impersonators are slow again

Sam and I identified parts of the reason why impersonators in bubblesort is very slow. We had a fix, but the introduction of caselambda broke this again. Needs investigation.

remainder

Add remainder and a working version of quotient/remainder.

Implement multiple values

earley.sch is slower than racket

[samth@huor:~/sw/pycket/pycket (master) plt] ./pycket.sh test/earley.sch 
cpu time: 2656 real time: 2656 gc time: 0
742900
[samth@huor:~/sw/pycket/pycket (master) plt] ./pycket.sh test/earley.sch 
cpu time: 2664 real time: 2664 gc time: 0
742900
[samth@huor:~/sw/pycket/pycket (master) plt] ./pycket.sh test/earley.sch 
cpu time: 2656 real time: 2656 gc time: 0
742900
[samth@huor:~/sw/pycket/pycket (master) plt] r -f test/earley.sch 
cpu time: 1960 real time: 1966 gc time: 1308
[samth@huor:~/sw/pycket/pycket (master) plt] r -f test/earley.sch 
cpu time: 1972 real time: 1974 gc time: 1316
[samth@huor:~/sw/pycket/pycket (master) plt] r -f test/earley.sch 
cpu time: 1972 real time: 1975 gc time: 1316

sumloop uses the environment

Sumloop fetches a fixnum from the environment every time. I tried speculating more in fa72bc1 but it didn't help.

Pointless check in bubblesort init

Right now, the initialization loop in bubblesort generates code like this:

i26 = int_lt(i18, i7)
guard_false(i26)
...
i25 = int_lt(i18, i9)
guard_false(i25)

i7 and i9 are both loop-carried variables that never change in the loop and are always 1000. Why are we doing both tests?

Implement chaperones/impersonators

Docs here: http://docs.racket-lang.org/reference/chaperones.html

Relies on a solution to #3

pycket-c crashes when primitive 'apply' is first-class (i.e. not applied directly)

This minimal program crashes pycket-c:

#lang pycket
apply

Here is the output:

RPython traceback:
  File "pycket_interpreter.c", line 21084, in portal
  File "pycket_interpreter.c", line 21510, in AST_interpret
  File "pycket_interpreter.c", line 28724, in ModuleVar__elidable_lookup
Fatal RPython error: AssertionError
Abort trap: 6

In the json file, apply has been expanded to new-apply-proc, which pycket-c does not expect:

{"module-name":"apply_min","body-forms":[
{"module":"new-apply-proc",
"source-module":"$RACKET/collects/racket/private/pre-base.rkt",
"source-name":"new-apply-proc"}]}

By eta-expanding apply, we get the json that pycket-c understands:

#lang pycket

(lambda (f args) (apply f args))

{"module-name":"apply_eta","body-forms":[[
{"module":"lambda","source-module":"#%kernel","source-name":"lambda"},
[{"lexical":"f"},{"lexical":"args"}],
[{"module":"#%app","source-module":"#%kernel","source-name":"#%app"},
{"module":"apply","source-module":"#%kernel","source-name":"apply"},
{"lexical":"f"},{"lexical":"args"}]]]}

The work-around of eta-expanding apply works well enough, but I just thought I'd report it because it took me quite a while to debug why a program of mine (based on tower of interpreters) ran in Scheme but not pycket.

The new-apply-proc is coming straightforwardly from Racket expansion, so I am not sure whether this is work as intended on the Racket side.

#lang racket

(pretty-print (syntax->datum
(parameterize ([current-namespace (make-base-namespace)])
 (expand
  (datum->syntax
   #f
   '(module foo scheme
      apply))))))

;; ==>
'(module foo scheme
   (#%module-begin
    (module configure-runtime '#%kernel
      (#%module-begin (#%require scheme/runtime-config) (#%app configure '#f)))
    (#%app call-with-values (lambda () new-apply-proc) print-values)))

Implement delimited continuations + dynamic wind

test_puzzle gets stuck

Running

 ~/sw/pycket/pypy/pytest.py test/jit.py -s -k test_puzzle

seems to get stuck.

lexer error on +inf.0

need to handle +inf.0 and -inf.0 as racket numbers.

Try to get branch reduce_prim_cont working

That would allow us to not set should_enter on calls to primitives.

bubble-unsafe.sch fails

With Racket, it works fine. With compiled pycket, it segaults. With pycket-slow.sh, it produces:

[samth@huor:~/sw/pycket/pycket (master) plt] ./pycket-slow.sh test/bubble-unsafe.sch
Traceback (most recent call last):
  File "app_main.py", line 72, in run_toplevel
  File "runpycket.py", line 18, in <module>
    main(sys.argv)
  File "runpycket.py", line 8, in main
    val = interpret_one(ast)
  File "/home/samth/sw/pycket/pycket/interpreter.py", line 736, in interpret_one
    ast, env, cont = ast.interpret(env, cont)
  File "/home/samth/sw/pycket/pycket/interpreter.py", line 373, in interpret
    return return_value(self._lookup(env), env, cont)
  File "/home/samth/sw/pycket/pycket/interpreter.py", line 235, in return_value
    return return_multi_vals(values.Values.make([w_val]), env, cont)
  File "/home/samth/sw/pycket/pycket/interpreter.py", line 240, in return_multi_vals
    return cont.plug_reduce(vals)
  File "/home/samth/sw/pycket/pycket/interpreter.py", line 182, in plug_reduce
    return vals_w[0].call(vals_w[1:], env, self.prev)
  File "/home/samth/sw/pycket/pycket/values.py", line 150, in call
    return return_value(self.code(args), env, cont)
  File "/home/samth/sw/pycket/pycket/prims.py", line 36, in wrap_func
    result = func(*args)
  File "/home/samth/sw/pycket/pycket/prims.py", line 462, in unsafe_vector_set
    return v.set(i.value, new)
  File "/home/samth/sw/pycket/pycket/vector.py", line 47, in set
    self.strategy.set(self, i, v)
  File "/home/samth/sw/pycket/pycket/vector.py", line 79, in set
    self.indexcheck(w_vector, i)
  File "/home/samth/sw/pycket/pycket/vector.py", line 86, in indexcheck
    assert 0 <= i < w_vector.len
AssertionError

Come up with a strategy to call functions from primitives

For example, implementing map as a primitive would be painful because of the CESK stack.

Implement parameters

Support multiple values

Improve gcbench

Figure out why gcbench is slow, given that the pypy version is rather fast on it.

More principled approach to environment specularion in W_Closure.call

We changed things to speculate on two levels of environment identity speculation (028e249). Can there be a more principled approach to do that other than increasing the levels? Is increasing the levels worth it?

Ideas:

do something based on environment types
do something using the static environment structure

test_setbang never jits

Running the jit.py test test_setbang appears never to produce jit code. At least, it never brings up the viewer.

support procedure arity information

We need to support both procedure-arity and procedure-arity-includes?. The latter is probably easier.

This requires including arity information on all exposed primitives.

make `time` accurate

Right now, we treat cpu time as the same as real time, and ignore gc time.

eval-jit-enabled should know about our JIT

Right now it always returns true.

unsafe ops do too many checks

@expose("unsafe-vector-length", [values.W_MVector])
def unsafe_vector_length(v):
    return values.W_Fixnum(v.length())

This actually checks if the provided value is a W_MVector.

nucleic2 and nucleic3 fail

[samth@huor:~/sw/pycket/pycket (master) plt] ./pycket.sh test/nucleic2.sch
RPython traceback:
  File "pycket_targetpycket.c", line 225, in main
  File "rpython_jit_metainterp_warmspot.c", line 103, in ll_portal_runner__pycket_interpreter_ASTPtr_pyck
  File "pycket_interpreter.c", line 10001, in portal
  File "pycket_values.c", line 5034, in W_SimplePrim_call
  File "pycket_prims.c", line 48382, in wrap_do_car__star_0
Fatal RPython error: SchemeException
Aborted (core dumped)
[samth@huor:~/sw/pycket/pycket (master) plt] ./pycket.sh test/nucleic3.sch
RPython traceback:
  File "pycket_targetpycket.c", line 235, in main
  File "pycket_json.c", line 313, in loads
  File "rpython_rlib_parsing_parsing.c", line 883, in PackratParser_parse
Fatal RPython error: ParseError
Aborted (core dumped)

Investigate slowdown from Dyla to now

These benchmarks: https://dl.dropboxusercontent.com/u/14917452/20140710_fast-norm-col.pdf which @krono made show us as now slower than Racket. When we submitted to Dyla, we were faster than Racket. We need to figure out what happened.