To reproduce:
{ok, P} = gen_tcp:listen(12321, [binary, {active,false}]).
{ok,Q} = gen_tcp:accept(P).
Telnet to port 12321.
Press return twice.

Observed result:
Port goes into a busy-loop; with inet debugging enabled, this is visible from the debug output.

Code analysis:
TCPInet.tcp_recv() handles the "socket is passive, buffer is no-empty" case badly; it does not read any input, nor does it disable interest.
In the above case,
i_remain==0, nread==0, lenp[0]==2 (for "\r\n")
which makes tcp_recv() try to deliver data (and return) without having read any new data. Hence the input remains ready.

An unused com.sun package import for a charset in EBigString breaks build in javac 1.6.0_06.

(Needed by module unicode)

When I did some load testing on mochiweb / erjang I got this exception. Have to look into it later.

Exception in thread "Thread-5" java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:69)
at sun.nio.ch.KQueueSelectorImpl.updateSelectedKeys(KQueueSelectorImpl.java:105)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:74)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
at erjang.driver.NIOSelector.run(NIOSelector.java:113)

==== To reproduce:
1> erlang:term_to_binary(fun erlang:is_lt/2).

=== Observed behaviour:
** exception error: {not_implemented,"Encode for erjang.EFunHandler2",
"erjang.EObject.encode(EObject.java:435)"}
in function term_to_binary/1

ECons.compare_same() breaks the double-dispatch pattern used elsewhere.
Is there a good reason for this?

Current implementation of match_spec() is an AST interpreter. Make a JVM codegen module for that.

java.util.NoSuchElementException
at java.util.LinkedList.remove(LinkedList.java:788)
at java.util.LinkedList.removeFirst(LinkedList.java:134)
at java.util.LinkedList.poll(LinkedList.java:470)
at kilim.Scheduler.schedule(Scheduler.java:67)
at kilim.Task.resume(Task.java:203)
at kilim.Task._runExecute(Task.java:448)
at kilim.WorkerThread.run(WorkerThread.java:47)
[java] exiting <0.0.4> with: java.util.NoSuchElementException
<0.0.4>4(running=true,pr=null)::erlang:apply/2
at java.util.LinkedList.remove(LinkedList.java:788)
at java.util.LinkedList.removeFirst(LinkedList.java:134)
at java.util.LinkedList.poll(LinkedList.java:470)
at kilim.Scheduler.schedule(Scheduler.java:67)
at kilim.Task.resume(Task.java:203)
at kilim.Task.messageAvailable(Task.java:178)
at kilim.Mailbox.put(Mailbox.java:318)
at kilim.Mailbox.put(Mailbox.java:411)
at erjang.driver.EDriverTask.close(EDriverTask.java:574)
at erjang.EInternalPort.close(EInternalPort.java:181)
at erjang.m.erlang.ErlPort.port_close(ErlPort.java:354)
at erjang.m.prim_file.prim_file.drv_close__1(prim_file.S:219)
at erjang.m.prim_file.prim_file$FN_drv_close__1.go(Unknown Source)
at erjang.m.prim_file.prim_file.close__1$call(prim_file.S)
at erjang.m.prim_file.prim_file.read_file__1(prim_file.S:96)
at erjang.m.prim_file.prim_file$FN_read_file__1.invoke(Unknown Source)

Hi,

I have just compiled Erjang, against Sun Java 1.6.0 and Erlang R14B01, on Ubuntu 10.10 64bit, but when I launch ej, it crashes with the following stack [1]. Any idea?

[1] http://pastebin.com/mmLSLMYb

In FDDriverInstance, in the run() function in the anonymous Thread class, EOF is handled by sending EOF signal to finish(), then allocating nother 1K buffer and repeating.
This busy loop means that on EOF, we have both a 100% CPU condition and (unless the receiver keeps up) a rapidly-growing-memory footprint issue.

...which means that an unconditional process termination is propagated unconditionally as well.
==== To reproduce:
1> process_flag(trap_exit, true).
false
2> link(spawn(fun()->timer:sleep(2000), exit(self(), kill) end)).
true

==== Observed behaviour:
3> ** exception error: killed
% I.e., initial process is slain as well.

==== Expected behaviour:
3> f(M), receive M->M end.
{'EXIT',<0.57.0>,killed}
% I.e., initial process receives a non-'kill' exit message.

CompilerVisitor generates such code for get_tuple_element when the argument is not statically known to be a tuple.

Modules for which such generated code has been seen: error_handler and yeccparser.
The assertion
assert(val.type.getInternalName().startsWith("erjang/ETuple"));
is triggered if inserted in the get_tuple_element case in CompilerVisitor.

(When fixing this, please also ensure that things also go right if get_tuple_element is used to extract element N from something which is statically only known to be a tuple of size M where M<N.)

I am quite certain that we have not covered all the bases for equals operations (and perhaps comparisons too), and it would be good to have a more complete test base for == and =:=.

We need to evaluate what java.lang.Object.equals() should be bound to, ==, =:= or something else. I think it needs to be bound to =:= (so that e.g. 0 and 0.0 are different in equals/hash maps).

Perhaps, to make things perfectly clear, EObject should have erl_equals(other) and erl_equals_exact(other); and then have EObject.equals(java.lang.Object) call one of those.

But before we can commence, we need some more consistent test.

The sync_port_sends branch has changes to make all port operations synchronous. In effect, this makes a port behave like a monitor (no message queue, all operations block on EDriverTask synchronized methods). Doing " ! msg" becomes a direct call into the driver.

I realized this is the needed behavior, as this is how BEAM implements ports. Problem is, that the mods in that branch breaks RabbitMQ booting, and I've not been able to figure out why. Would be great to have some unit tests for port behaviors.

This is a rather challenging problem, but I think that we can make an approximation. We need to have some logic implemented in Kilim to get a real stack trace (with class references) for another Kilim process. kilim.Task.getStackTrace() -> java.lang.Class[], which must work for processes that are both running, and processes that are dormant. For now, the implementation of erlang:check_process_code simply returns true.

It seems that even though we succeed loading and compiling io, the function fwrite/2 is not there anyway? This is strange. The module compiles and loads with no complaints; perhaps a race condition?

resolving io:fwrite/2
{get_file,"./io.beam"}
{return,{error,enoent}}
{get_file,"/sw/lib/erlang/lib/kernel-2.13.3/ebin/io.beam"}
{return,{error,enoent}}
{get_file,"/sw/lib/erlang/lib/stdlib-1.16.3/ebin/io.beam"}
{return,{ok,<<70,79,82,49,0,0,60,208,66,69,65,77,65,116,111,109,0,0,2,174,...,0>>}}
{get_cwd,[]}
{return,{ok,"/Users/krab/Documents/humio/erlang"}}
[compiling io:98ms]exiting PID<:351> with: erjang.ErlangRaise: {undef,[{io,fwrite,[<<69,115,104,101,108,108,32,86,126,115,32,32,40,97,98,111,114,116,32,119,...,10>>,["5.7.3"]]},{shell,server,1},{erlang,apply,2}]}
at erjang.ERT.raise(ERT.java:62)
at erjang.ERT.do_raise(ERT.java:48)
at erjang.m.error_handler.error_handler.crash__1(Unknown Source)
at erjang.m.error_handler.error_handler$FN_crash__1.go(Unknown Source)
at erjang.m.error_handler.error_handler$FN_undefined_function__3.invoke(Unknown Source)
at erjang.EFun3.invoke(Unknown Source)
at erjang.EFun.apply(EFun.java:422)

To reproduce

Eshell V5.7.5  (abort with ^G)
1> self().
<0.33.0>
2> {ok, P} = gen_tcp:listen(12329, [binary, {active,false}]).
{ok,#Port<0.513>}
3> {ok,Q} = gen_tcp:accept(P).
%% Telnet to the port at this point.
{ok,#Port<0.514>}
4> exit(out).
** exception exit: out
5> self().
<0.38.0>

Expected behaviour

The socket is closed at process exit (which can be observed in that it makes telnet terminate).

Observed behaviour

The socket survives process exit (telnet is oblivious of the process's demise).

== To reproduce:
{ok, P} = gen_tcp:listen(12321, [binary, {active,false}]).
{ok,Q} = gen_tcp:accept(P).
%% telnet to localhost:12321 at this point; don't send anything.
gen_tcp:recv(Q, 0).
%% make telnet disconnect

== Expected behaviour:
recv() returns with result {error,closed}.

== Observed behaviour:
recv() doesn't return.

== To reproduce:
{ok, P} = gen_tcp:listen(12321, [binary, {active,false}]).
{ok,Q} = gen_tcp:accept(P).
%% telnet to localhost:12321 at this point; don't send anything.
gen_tcp:recv(Q, 0, 250).
%% wait for 1 second

== Expected behaviour:
The result {error,timeout} after 250ms

== Observed behaviour:
Timeout is not in effect. Erjang waits for input on the socket.
Closing the connection from the telnet side doesn't even make recv() return.

To reproduce:

(fun erlang:integer_to_list/1)(1,0).

Expected behaviour:

** exception error: integer_to_list/1 called with two arguments

Observed behaviour:

Expression evaluates to "1".

EBitStringBuilder mixes byte-indexing and bit-indexing, and the core model must be bit indexed because a bit string builder can have individual bits added to it. Sadly, we need to redo most of the methods similar to the way it is done in EBitStringMatchState.

This is primarily a problem in connection with the interpreter; the compiler calls the guard functions directly without wrapping them in EFun's.

Option 1: Introduce opcodes for all guard functions (and probably other builtins as well).
Option 2: Make a special, non-tail-optimizing version of generated EFun's.
Temporary workaround: Have TAIL_MARKER = new ETailMarker().

Calls to ERT.decode_exception2() or ERT.decode_exception3() trigger stack trace decoding, which is rather costly when the stack is deep, such as in list comprehensions.

Currently EBitString has a "long bits" field, counting the total number of bits in a bitstring. This needs to be refactored into an "int octets" and "int extrabits", where extrabits is a number in the range 0..7.

BEAM/erts does not generate an error for this, Erjang does...

1> io:format(user, "~-*s", [45, ".#zz"]).
** exception exit: {badarg,[{io,format,[user,"~-*s",[45,".#zz"]]},
                        {io,o_request,3},
                        {erlang,apply,3},
                        {erl_eval,do_apply,5},
                        {shell,exprs,6},
                        {shell,eval_exprs,6},
                        {shell,eval_loop,3},
                        {erlang,apply,3}]}
 in function  apply/3
 in call from erl_eval:do_apply/5
 in call from shell:exprs/6
 in call from shell:eval_exprs/6
 in call from shell:eval_loop/3
 in call from apply/3

How to reproduce

process_flag(trap_exit, true), try exit(x), still_here catch E -> {caught,E} after io:format("after\n") end.

Expected

1> process_flag(trap_exit, true), try exit(x), still_here catch E -> {caught,E} after io:format("after\n") end.
after
** exception exit: x
2>

Observed

(rabbit@mcilroy)4> process_flag(trap_exit, true), try exit(x), still_here catch E -> {caught,E} after io:format("after\n") end.
after
java.lang.Throwable: bad argument to raise2: (x, [{erl_eval,expr,3}])
at erjang.ERT.raise(ERT.java:62)
at erjang.m.erl_eval.erl_eval.try_clauses__8(erl_eval.S:263)
at erjang.m.erl_eval.erl_eval$FN_try_clauses__8.go(Unknown Source)

I suspect we spend a lot of time loading function stubs for all function (specialized java.lang.reflect.Method-like things) when Erjang launches. Establish the truth in that statement; and then figure out if we can somehow make that lazy.

Every exported function needs one of these stubs (some subclass of EFun), but perhaps we can install a generic handler which will then replace itself when invoked the first time. Ideally, we should only need to load 1 class per module (until you start calling functions).

CompilerVisitor.push_immediate() generates bad code for io_lib_format:fwrite_g/1; the code contains a reference to field "num_0" of type EDouble. The field exists, but has type ESmall.
This bug is caused by collision between ESmall(0) and EDouble(0.0) in the constants map maintained by push_immediate().

How to reproduce

This appears to be a somewhat timing- and size-dependent issue, but the following steps appear to reproduce it consistently:
./ej
> {ok, P} = gen_tcp:listen(12321, [binary, {active,false}]),
{ok,Q0} = gen_tcp:accept(P),
{ok,Q} = gen_tcp:accept(P),
gen_tcp:recv(Q, 1),
gen_tcp:recv(Q, 4).
In another terminal:
echo | telnet localhost 12321 # Q0
(echo 'a'; sleep 1; echo 'b') | telnet localhost 12321 # Q

Observed result

(with -Derjang.debug.inet=true):
infinite debug output:

readyInput TCPIP@32425470[sock=InetClientSocket[java.nio.channels.SocketChannel[connected local=/127.0.0.1:12321 remote=/127.0.0.1:57780] open=true addr=/127.0.0.1:12321 remote=/127.0.0.1:57780]; state=AcBoConCogLiMuOp; active=PASSIVE; deliver=1; select=0; nointrest] @ java.nio.channels.SocketChannel[connected local=/127.0.0.1:12321 remote=/127.0.0.1:57780]
tcp_recv len=0
tcp_recv i_remain == 0
tcp_deliver 0:1
deliver.2
deliver.2.2
tcp_reply_data len=1
tcp_deliver::error(code) -1
tcp_recv[async] =>-1
readyInput TCPIP@32425470[sock=InetClientSocket[java.nio.channels.SocketChannel[connected local=/127.0.0.1:12321 remote=/127.0.0.1:57780] open=true addr=/127.0.0.1:12321 remote=/127.0.0.1:57780]; state=AcBoConCogLiMuOp; active=PASSIVE; deliver=1; select=0; nointrest] @ java.nio.channels.SocketChannel[connected local=/127.0.0.1:12321 remote=/127.0.0.1:57780]

Expected result

The port shuts down cleanly.

One way to solve this is to have EModuleManager make these dummies on demand, but only if configured to do so (i.e. it should know of a set of module names for which such dummies might be created).

Find inspiration here: http://github.com/jruby/jruby/blob/master/src/org/jruby/demo/IRBConsole.java

After compiling Erlang version R14B and setting my environment variables to:

ERTS_VSN=5.8.1

ERL_ROOT=/home/robert/bob-apps/erlang-for-erjang-R14B/lib/erlang

When I run ./ej i receive the following:

-

robertlj@rj:~/bob-repos/erjang_working$ ./ej
Nov 10, 2010 10:45:53 AM erjang.EModuleManager$ModuleInfo warn_about_unresolved
INFO: unresolved after load: code:is_module_native/1
Eshell V5.8.1 (abort with ^G)
1>

-

-

System OS: Ubuntu 10.10 (Maverick Meerkat)
Erlang: compiled from erlang.org source

Example of my "env_cfg"

-

if [[ -z ${ERTS_VSN} ]]; then
ERTS_VSN=5.8.1
fi
if [[ -z ${ERL_ROOT} ]]; then
ERL_ROOT=/home/robert/my-apps/erlang-for-erjang-R14B/lib/erlang
fi

-

Hi,
I have just tried to compile CouchDB 0.10, and was a success. Congratulations !!!

Is there a mailing list to discuss Erjang ?

Andrey

Of 2^16+1 different EFun instances, at least one pair will compareSame()==0.
(And according to the Birthday Paradox, a collision can be found among far fewer.)

The optimization in question makes certain non-tail recursions be rewritten so that they run in fixed stack space.
At present, io_lib:write_binary_body/2 sometimes dies because of Erjang lacking this feature.
Another example is lists:flatten applied to a sufficiently deeply nested list.

This shoudl not happen, as all out Sockets are created in non-blocking mode. Figure out why Java thinks it's a blockling connect -- which may cause time out exception. Non-blocking connect requests should fail by returning false.

unknown exception: java.net.ConnectException: Operation timed out
java.net.ConnectException: Operation timed out
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at erjang.net.InetClientSocket.finishConnect(InetClientSocket.java:113)
at erjang.net.LazyInetSocket.finishConnect(LazyInetSocket.java:158)
at erjang.driver.tcp_inet.TCPINet.readyConnect(TCPINet.java:1364)
at erjang.driver.LockingDriverInstance.readyConnect(LockingDriverInstance.java:194)
at erjang.driver.EDriverTask$3.execute(EDriverTask.java:642)
at erjang.driver.EDriverTask.main_loop(EDriverTask.java:459)

==== Current behaviour:
1> erlang:term_to_binary(fun erlang:'not'/1).
** exception error: {not_implemented,"Encode for erjang.m.erlang.ErlBif$FN_not__1",
"erjang.EObject.encode(EObject.java:435)"}

==== Desired behaviour:
Function objects for exported functions should be serialized using EOutputStream.write_external_fun(module, function, arity).

To reproduce: evaluate
{ets:new(foo,[named_table]), ets:new(foo,[named_table])}.
Observed result:
{foo,foo}
Expected result:
{foo,{'EXIT',{badarg,[{ets,new,[foo,[named_table]]} | _}}}

Check out pages.github.com, and create a gh-pages site for erjang.org.

I would like there to be the javadoc or doxygen or something that makes it easy for people to browse the source / api to better understand it.

These BIFs (or at least one of them) is necessary for shell to work.

The code:
exn() -> try erlang:error(x)
catch E -> {caught,E}
after io:format("after\n")
end.

results in this error:
l(exntest).
13-11-2010 10:36:10 erjang.m.erlang.ErlBif load_module
SEVERE: cannot load module exntest
java.lang.RuntimeException: Verify error. Expected word in stack, but stack is empty
at kilim.analysis.Frame.pop(Frame.java:231)
at kilim.analysis.Frame.popWord(Frame.java:236)
at kilim.analysis.BasicBlock.interpret(BasicBlock.java:613)
at kilim.analysis.MethodFlow.dataFlow(MethodFlow.java:374)
at kilim.analysis.MethodFlow.analyze(MethodFlow.java:126)
at kilim.analysis.ClassFlow.analyze(ClassFlow.java:104)
at kilim.analysis.ClassWeaver.weave(ClassWeaver.java:59)
at kilim.analysis.ClassWeaver.(ClassWeaver.java:45)
at erjang.beam.Compiler.compile(Compiler.java:104)

In the beam->jvm codegen, I think we can make the code gen more efficient by making a separate pass on an entire module to determine which functions need to be Pausable.

All functions are encoded as static members of the module class, and non-external calls between functions in the same module thus are direct static calls. Now, if we can determine methods (typically leafs in the call graph) which do not call external functions, and which do not use BIFs that are Pausable, and make those functions also be non-Pausable, then it would reduce the overhead generated by Kilim significantly; and potentially speed up things quite significantly.

Such a pass needs to build a dependency graph containing all intra-module calls, and scan functions for BIFs or external calls. The "Pausable"ness then needs to flow backwards in that dependency graph.

EFile is using blocking IO to send messages back to Erlang. If the mailbox runs full, and there are no threads available to receive the messages we may deadlock.

Solutions: 1) make EDriverInstance.output_from_driver be Pausable, or 2) make a separate thread pool for IO, so that we can always assume that someone will eventually take the output.

Since enabling the new shutdown code, when doing erlang:halt() from the shell, it will spew ugly error messages. Also, hitting ^D (EOF) will make the shell restart for ever.

It may (and may not) be directly related to the refactoring done in this commit b5147c8

As it is now, paths to OTP, erl and erlc are hard-coded in the files src/test/java/erjang/AllTests.java, src/test/java/erjang/TestRunFile.java.
I suggest creating a directory ext/ for holding symlinks to these external entities; or, if that won't work for Windows users, that the paths in question are put into appropriate Java properties and set in build.xml.

Codegen needs to call ERT.resolve, and cast the result to EFunX, and call directly.

This needs to be fixed in CompilerVisitor where there is a usage of BeamOpcode.apply.

To reproduce:

{ok, P} = gen_tcp:listen(12321, [binary, {active,false}]).
{ok,Q} = gen_tcp:accept(P).
%% telnet to localhost:12321 at this point; don't send anything.
gen_tcp:recv(Q, 0).
%% make telnet disconnect

Expected behaviour:

recv() returns with result {error,closed}.

Observed behaviour:

recv() doesn't return.

For some reason, floats are printed incorrectly in eshell. Problem has something to do with interpretation of io_lib_format; if someone wants to hunt it down that would be great.

./erl.sh 
Eshell V5.7.3  (abort with ^G)
1> 1.0.
0.00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010261342003245941

Kilim reports this error on the asn1rt_uper_bin compile test case:
"erjang.m.asn1rt_uper_bin.getbits_as_list__2(Lerjang/EProc;Lerjang/EObject;Lerjang/EObject;)Lerjang/EObject; should be marked pausable. It calls pausable methods erjang.m.asn1rt_uper_bin.$2Dgetbits_as_list$2F2$2Dblc$$$5E0$2F1$2D0$2D__1(Lerjang/EProc;Lerjang/EObject;)Lerjang/EObject;
(and other messages like it).