jgpc42 / insn Goto Github PK

View Code? Open in Web Editor NEW

193.0 12.0 5.0 276 KB

Functional JVM bytecode generation for Clojure.

License: Eclipse Public License 1.0

Clojure 98.57% Shell 1.01% Dockerfile 0.42%

clojure jvm-bytecode asm bytecode-instructions

insn's People

Contributors

Stargazers

Watchers

Forkers

bhugueney phronmophobic derrickzeo 00mjk edipofederle

insn's Issues

I love this library

Just wanted to say so publicly and officially.

We use it at a pretty deep level to build optimized dimension-specific indexing systems for ND operations:

https://github.com/techascent/tech.datatype/blob/57988eef8e544b02f6f36a1b6f52aef3a02cb0d1/src/tech/v2/tensor/dimensions/global_to_local.clj

I wrote a sort of blog post about my experience using it a while ago and meant to post it here:

https://github.com/techascent/tech.datatype/blob/57988eef8e544b02f6f36a1b6f52aef3a02cb0d1/docs/dimensions-bytecode-gen.md

try finally

How would you do a general try/finally pattern?

Looking at bytecode for a simple case yields:

chrisn@chrisn-lt-01:~/dev/cnuernber/dtype-next$ javap -c java/tech/v3/datatype/TryFinally.class
Compiled from "TryFinally.java"
public class tech.v3.datatype.TryFinally {
  public tech.v3.datatype.TryFinally();
    Code:
       0: aload_0
       1: invokespecial #1                  // Method java/lang/Object."<init>":()V
       4: return

  public static int testfn(int);
    Code:
       0: iconst_4
       1: istore_0
       2: new           #7                  // class java/lang/RuntimeException
       5: dup
       6: ldc           #9                  // String Exception
       8: invokespecial #11                 // Method java/lang/RuntimeException."<init>":(Ljava/lang/String;)V
      11: athrow
      12: astore_1
      13: iconst_5
      14: istore_0
      15: aload_1
      16: athrow
    Exception table:
       from    to  target type
           0    13    12   any
}

The exception table in this case as target type any which I guess I could use Throwable for.  The trycatch instruction is a bit confusing to me I have to admit.

Thanks for any help!

Question: how to express call to default method of interface

Given the following example:

(ns user)

(require '[insn.core :as insn])

(def class-data
  {:name 'my.pkg.Iterator
   :version 1.8
   :interfaces ['java.util.Iterator]
   :fields [{:flags #{:public :static}, :name "VALUE", :type :boolean, :value true}]
   :methods [{:flags #{:public}, :name "hasNext"
              :desc [:boolean]
              :emit [[:getstatic :this "VALUE" :boolean]
                     [:ireturn]]}
             {:flags #{:public}, :name "remove"
              :emit [[:aload 0]
                     [:invokespecial java.util.Iterator :remove [:void]]
                     [:return]]}]})

(def result (insn/visit class-data))

(def class-object (insn/define class-data))

(-> class-object .newInstance (.hasNext))

The last expression returns true. But when I call the remove method, which is supposed to call the default method of the interface, I get:

(-> class-object .newInstance (.remove))

1. Unhandled java.lang.IncompatibleClassChangeError
   Method 'void java.util.Iterator.remove()' must be InterfaceMethodref constant

In Java this would be:

  java.util.Iterator.super.remove();

How can I write this?

Static invocation inside if branch, possible issue or just doing wrong?

Hi,

First of all, thanks for your work on this library!

I'm facing some issues when trying to invoke static inside a if branch, here be example:

(def test-data
  {:name "my.pkg.Test.Test",
   :flags [:public],
   :fields [],
   :methods
   [{:flags #{:public},
     :name "invoke",
     :desc [java.lang.Long :void],
     :emit
     [[:aload 1]
      [:invokevirtual java.lang.Long "longValue" [:long]]
      [:ldc2 100]
      [:lcmp]
      [:ifle "MARK1"]
      [:new "my.pkg.Test.Println"]
      [:dup]
      [:invokespecial "my.pkg.Test.Println" :init [:void]]
      [:astore 2]
      [:aload 2]
      [:aload 1]
      [:invokevirtual "my.pkg.Test.Println" "invoke" [java.lang.Object Object]]
      [:mark "MARK1"]
      [:return]]}]})
;;; Trying it: 

  (-> (insn/define test-data) .newInstance (.invoke 1000))

This is the Exception:

1. Unhandled java.lang.ArrayIndexOutOfBoundsException
   Index 0 out of bounds for length 0

                Frame.java: 1268  org.objectweb.asm.Frame/merge
                Frame.java: 1244  org.objectweb.asm.Frame/merge
         MethodWriter.java: 1611  org.objectweb.asm.MethodWriter/computeAllFrames
         MethodWriter.java: 1547  org.objectweb.asm.MethodWriter/visitMaxs
                  core.clj:  246  insn.core/visit-method
                  core.clj:  215  insn.core/visit-method
                  core.clj:  196  insn.core/visit/fn
                  core.clj:  191  insn.core/visit
                  core.clj:   42  insn.core/visit
                  core.clj:  266  insn.core/ensure-visited
                  core.clj:  265  insn.core/ensure-visited
                  core.clj:  279  insn.core/define
                  core.clj:  274  insn.core/define
                  core.clj:  277  insn.core/define
                  core.clj:  274  insn.core/define

I have basically this code in Java and the byte code looks like this, and works as expected:

   
  0: aload_1
       1: invokevirtual #7                  // Method java/lang/Long.longValue:()J
       4: ldc2_w        #13                 // long 100l
       7: lcmp
       8: ifle          24
      11: new           #15                 // class Println
      14: dup
      15: invokespecial #17                 // Method Println."<init>":()V
      18: astore_2
      19: aload_2
      20: aload_1
      21: invokevirtual #18                 // Method Println.invoke:(Ljava/lang/Object;)V
      24: return

I'm not not sure if I'm doing some thing wrong on the the Clojure code, or if maybe there is some bug in the library. Maybe someone can provide some direction here?

Thanks!

I'm looking to write a Clojure lib to allow for writing Azure Functions purely in Clojure. The Azure Function framework uses reflection to lookup values of parameterized types and uses that information to convert payloads etc to the correct type when calling the method. As Clojure doesn't emit type signatures Azure doesn't work.

I've been playing around with insn, my idea being that I create the class with the correct type signatures and then hopefully Azure will be happy. However, my understanding is that 'desc' values only use the base type and all generic information is stored under the type signature. Is it currently possible to write signature information and if so would it be possible to grief example?

Much appreciated.

Clojure AOT compilation bug

Currently, Clojure AOT compiling does not work with "dynamic" class imports like the ones used in this library.

The bug is shown here. I guess I'll report it if I get the time, but looking at the amount of open AOT-related bugs, I don't think it will be a high-priority fix.

I'm open to suggestions on how to handle this. The only thing that I can think of currently is to remove the ability to choose the ASM version you want to use.

Default constructor for Record?

I got curious and tried emitting a record with the library.
The awesome part is it just worked. The "hard" part was it did not emit the default constructor emitted by javac, so I had to write one myself.
Looking at some bytecode, generating it seems straightforward:

(defn emit-record-constructor
  [fields]
  (conj
   (into
    [[:aload 0]
     [:invokespecial :super :init [:void]]]
    cat
    (map-indexed
     (fn [i {:keys [name type]}]
       [[:aload 0]
        [:aload (inc i)]
        [:putfield :this name type]])
     fields))
   [:return]))

Do you think it should be added by default when no init is provided with the same desc? (when the class's flags have :record)

Minimal example getting errors (newb)

Thanks for this library, it's really opened up my eyes toward custom jvm bytecode magic.

I'm just playing around with some simple class definitions. One I'd like is to work with a primitive array, and int index, and a primitive value, and update the array at index with the value (basically a clone of clojure.core/aset without any int cast).

If I write a minimal java class, I get this:

public class demo
{public static char aset(char [] arr, int idx, char v)
  {arr[idx] = v;
   return v;
   }}

public class blah.demo
{public blah.demo();
 Code:
 0: aload_0
 1: invokespecial #1                  // Method java/lang/Object."<init>":()V
 4: return

 public static char aset(char[], int, char);
 Code:
 0: aload_0
 1: iload_1
 2: iload_2
 3: castore
 4: iload_2
 5: ireturn
 }

Translating to insn, I come up with:

(ns ultrarand.bytecode
  (:require [insn.core :as insn]))

(def class-data
  {:name    'my.pkg.Adder
   :flags  #{:public}
   :fields  []
   :methods [{:flags #{:public :static}, :name "asetLiteral", :desc [chars :int :char]
              :emit [[:aload 0]
                     [:iload 1]
                     [:iload 2]
                     [:castore]
                     [:iload 2]
                     [:ireturn]]}]})

Then I get a complaint from the classloader regarding incompatibility of types:

ultrarand.bytecode> (def class-object (insn/define class-data2))
ultrarand.bytecode> (def c (.newInstance class-object))
Execution error (VerifyError) at java.lang.Class/getDeclaredConstructors0 (Class.java:-2).
Bad local variable type
Exception Details:
  Location:
    my/pkg/Adder.asetLiteral([CI)C @2: iload_2
  Reason:
    Type top (current frame, locals[2]) is not assignable to integer
  Current Frame:
    bci: @2
    flags: { }
    locals: { '[C', integer }
    stack: { '[C', integer }
  Bytecode:
    0x0000000: 2a1b 1c55 1cac

If I try to dumb it down to the simplest possible return, a method that takes an int array, an int, and an int, and returns the last arg (int):

public class blah.demo
{public blah.demo();
 Code:
 0: aload_0
 1: invokespecial #1                  // Method java/lang/Object."<init>":()V
 4: return

 public static int aset(int[], int, int);
 Code:
 0: iload_2
 1: ireturn}

(def class-data2
  {:name    'my.pkg.Adder
   :flags  #{:public}
   :fields  []
   :methods [{:flags #{:public :static}, :name "asetLiteral", :desc [ints :int :int]
              :emit [[:iload 2]
                     [:ireturn]]}]})

ultrarand.bytecode> (def class-data2
  {:name    'my.pkg.Adder
   :flags  #{:public}
   :fields  []
   :methods [{:flags #{:public :static}, :name "asetLiteral", :desc [ints :int :int]
              :emit [[:iload 2]
                     [:ireturn]]}]})
#'ultrarand.bytecode/class-data2
ultrarand.bytecode> (def class-object (insn/define class-data2))
#'ultrarand.bytecode/class-object
ultrarand.bytecode> (def c (.newInstance class-object))
Execution error (VerifyError) at java.lang.Class/getDeclaredConstructors0 (Class.java:-2).
Bad local variable type
Exception Details:
  Location:
    my/pkg/Adder.asetLiteral([II)I @0: iload_2
  Reason:
    Type top (current frame, locals[2]) is not assignable to integer
  Current Frame:
    bci: @0
    flags: { }
    locals: { '[I', integer }
    stack: { }
  Bytecode:
    0x0000000: 1cac

Am I missing something in translating the bytecode to insn? It seems straightforward enough...

Platform:

Windows 10,
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_222-b10)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.222-b10, mixed mode)

Dependencies: (the insn and asm stuff was cloned from tech.datatype, so I may be out of date!)
[[org.clojure/clojure "1.10.1"]
[com.clojure-goes-fast/clj-java-decompiler "0.3.0"]
[insn "0.4.0"
:exclusions [org.ow2.asm/asm]]
[org.ow2.asm/asm "7.1"]]

Support for setting the SourceFile attribute of the class

I am getting a crash in graalvm's system because the sourcefile attribute isn't set on a generated class.

Is there a way I can set the SourceFile attribute? It appears you have to visit the ClassNode in asm2:

https://javadoc.io/doc/org.ow2.asm/asm/5.2/org/objectweb/asm/tree/ClassNode.html#visitSource-java.lang.String-java.lang.String-

How would you recommend setting the sourcefile attribute?

Upgrading to 0.5.3 causes `Constant pool index 75 is invalid`

Hi @jgpc42 - I upgraded to 0.5.3 for the new support for the invokespecial of a default interface method. But doing so broke generation of the classes that worked using 0.5.2. When constructing such a class, I'm getting errors like:

Syntax error (VerifyError) compiling new at (build/reify2.clj:171:1).
Illegal type at constant pool entry 75 in class babashka.impl.clojure.lang.IFn
Exception Details:
  Location:
    babashka/impl/clojure/lang/IFn.invoke(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; @7: invokeinterface
  Reason:
    Constant pool index 75 is invalid
  Bytecode:
    0000000: 2ab4 0036 b200 24b9 004b 0200 c000 063a
    0000010: 0b19 0bc6 001d 190b 2a2b 2c2d 1904 1905
    0000020: 1906 1907 1908 1909 190a b900 5f0c 00b0
    0000030: 2a2b 2c2d 1904 1905 1906 1907 1908 1909
    0000040: 190a b700 87b0                         
  Stackmap Table:
    append_frame(@48,Object[#6])

I tried downgrading ASM to 9.0 but that wasn't the issue.

The code to repro:

https://github.com/babashka/babashka/blob/72efde064e4e62ae847999479df31f35092c9588/reify/build/reify2.clj

How to use labels?

Hey, still really enjoying this library!

I am working on JDK16 support for dtype-next.

I am curious how to use the goto/if statements; specifically if-icmpne. How do I label a section of code?

static initializer section

I may have missed it but how would you recommend I code up a static initialization section?

saving out .class files

Hey, I am working with JDK-16 and I would like to statically generate some classes and load them back via import.

I tried saving the result of get-bytes verbatim into a .class file and the result is the importer thinks the class file is truncated. Do you have an example or any thoughts about this pathway?

I need to be able to save these classes so that I can work with systems such as Graal.