pubref / rules_kotlin Goto Github PK

see this. While investigating to see if the background workers add any performance gains after updating rules to follow the worker protocol protocol (see #54) I disovered that compiling erroneous code in error mode from a persistent workers leads to:

---8<---8<--- Start of log, file at /private/var/tmp/_bazel_hassan/b35d318b00182129bd971b30e0d6ece1/bazel-workers/worker-4-KotlinCompile.log ---8<---8<---
(empty)
---8<---8<--- End of log ---8<---8<---
INFO: Elapsed time: 8.683s, Critical Path: 7.58s

The compiler processes seem to write to stdout / stderr instead of log files that Bazel allocates. This doesn't work for the intellij plugin or when the process is in background mode.

When building kotlin_binary with Bazel 0.8.1, I get this error:

ERROR: C:/users/achung/appdata/local/temp/_bazel_achung/uzq-tc6q/external/org_pubref_rules_kotlin/java/org/pubref/rules/kotlin/BUILD:1:1: Building external/org_pubref_rules_kotlin/java/org/pubref/rules/kotlin/worker.jar (1 source file) and running annotation processors (ComponentProcessor) failed (Exit 1)
external\org_pubref_rules_kotlin\java\org\pubref\rules\kotlin\KotlinWorker.java:18: error: cannot access org.pubref.rules.kotlin
package org.pubref.rules.kotlin;
^
  /io/bazel/rules/closure/BazelWorker_Factory.class

Hi, thanks for adding the java_deps attribute! It's working for the straightforward case of a java_library built from java sources, but doesn't seem to be working for the case of a java_library which exports other jars.

Here are steps to reproduce using examples/helloworld:

1. Add the following to the WORKSPACE file:

maven_jar(
    name = "com_google_guava_guava_21_0",
    artifact = "com.google.guava:guava:jar:21.0",
)

2. Add the following to examples/helloworld/BUILD:

java_library(
    name = "guava",
    exports = [
        "@com_google_guava_guava_21_0//jar",
    ],
)

3. In the same BUILD file, just to show it working for java rules, add ":guava" to the deps attribute of the java_binary named "main_java", like so:

java_binary(
    name = "main_java",
    main_class = "examples.helloworld.Main",
    srcs = ["Main.java"],
    deps = [":rules_kt", ":guava"],
)

4. Also to show it working for java, add the following import line to Main.java:

import com.google.common.base.Joiner;  // This compiles.

5. Then, in the BUILD file, add ":guava" to the java_deps attribute of the kotlin_binary named "main_kt", like so:

kotlin_binary(
    name = "main_kt",
    main_class = "examples.helloworld.MainKt",
    srcs = ["main.kt"],
    deps = [":rules"],
    java_deps = [":milk", ":guava"]
)

6. Up to this point, observe the following command works:

bazel build //examples/helloworld:all

7. However, this step causes a failure to build: Add the following import line to main.kt:

// This import fails to resolve.
import com.google.common.base.Joiner

The failure is:

rules_kotlin/examples/helloworld/main.kt:3:12: error: unresolved reference: google
import com.google.common.base.Joiner
           ^

The same problem happens with the kotlin_library rule. I'm using bazel 0.4.5, and rules_kotlin cloned at 522a1e7.

Hi!

First of all: thanks for your great work :)

I've tried creating a project according to the documentation, but I cannot run the resulting jar file. It prints:

Exception in thread "main" java.lang.NoClassDefFoundError: kotlin/jvm/internal/Intrinsics at test.MainKt.main(Main.kt) Caused by: java.lang.ClassNotFoundException: kotlin.jvm.internal.Intrinsics at java.net.URLClassLoader$1.run(URLClassLoader.java:372) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:360) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 1 more

Here is a minimal project I'm trying to run: RulesKotlinMinimal.zip

UPDATE This fixed my problem: https://github.com/plumpy/rules_kotlin/commit/54158aa6ee0d4fd1a07867a72c721316511fe05b

The current kotlin_library rule seems to be limited to pure kotlin library, I cannot find a way to use it to import Android libraries into kotlin (something like: https://kotlinlang.org/docs/tutorials/kotlin-android.html)

Is there any support for something similar to the java_plugin rule? I would like to use kotlin with dagger2 and require that the kotlin rules plug in to the kapt functionality.

Loading the kotlin_binary rule triggers the build of some native libraries (protobuf, maybe others). It's a slightly time consuming step, so I am wondering if this step is necessary -- I haven't compiled any proto with Bazel yet so I don't know what the other language rules offering protobuf support do.

An example is at https://github.com/fishy/onepwd/tree/c56928e42bc521df2be9f975958019718dae53f5. In https://github.com/fishy/onepwd/blob/c56928e42bc521df2be9f975958019718dae53f5/BUILD#L40-L50, only com_android_support_appcompat_v7_27_0_1 and com_android_support_preference_v7_27_0_1 are actually directly depended, and everything else in aar_deps are just transitive dependencies from those 2 libraries (you can see that on https://raw.githubusercontent.com/aj-michael/gmaven_rules/master/gmaven.bzl).

But if I remove everything else and only kept the 2 libraries, bazel build :onepwd_compile will fail, indicates that the transitive dependencies are not passed to the kotlin_compile rule correctly.

https://github.com/bazelbuild/intellij/issues/148

@pcj The bazel IJ plugin works fairly without any config, it does wipe the Kotlin configuration on each workspace sync. It would be great to contribute basic Kotlin support to the IJ plugin codebase.

I'll have a gander at the task but I don't have the bandwidth at the moment. I can help test.

Doc: https://kotlinlang.org/docs/reference/multiplatform.html

I think there is something great we can achieve here to marry the multiplatform feature from [email protected] with Bazel to support Write once, Compile everywhere ambition. For example, the workflow of a full stack development would be: 1) write shared business logic libraries in Kotlin 2) write platform-specific backend/microservice code in Kotlin 3) write mobile/desktop/browser client side application logic code in Kotlin 4) bazel test & bazel build

I wonder whether there is anyone at Pubref interested in discussing and experimenting implementation for that? Based on git history, @pcj any thought ^^

Thanks for sharing your kotlin rules!

I'm trying to use a java_library from a kotlin_library, but it doesn't seem to be working for me.

Steps to reproduce the issue using the hello world example:

1. Clone kotlin_rules repo at commit 2e52fb9.
2. Build all works initially:

$ bazel build --strategy=KotlinCompile=standalone //examples/helloworld:all

3. Add the following jars line to examples/helloworld/BUILD:

kotlin_library(
    name = "rules",
    srcs = ["rules.kt"],
    jars = [":milk"]  # Added this line.
)

4. Now build all fails:

$ bazel build --strategy=KotlinCompile=standalone //examples/helloworld:all
ERROR: /tmp/rules_kotlin_clone/examples/helloworld/BUILD:25:1: in jars attribute of java_import rule //examples/helloworld:rules_kt: should not refer to Java rules. Since this rule was created by the macro 'kotlin_library', the error might have been caused by the macro implementation in /tmp/rules_kotlin_clone/kotlin/rules.bzl:200:16.
ERROR: Analysis of target '//examples/helloworld:rules_kt' failed; build aborted.

I'm using bazel version 0.4.4.

Suppose there is a target like this:

kotlin_library(
    name = "core",
    visibility = ["//visibility:public"]
    ...
)

Then the implicit output target core_kt will still have default visibility.
It should have inherited "//visibilty:public" however.

I checked that this is what the java rules do.

Thanks for making these cool Kotlin rules for Bazel!

I was playing around with this in my aaronj1335/bazel-kotlin-android repo, but when I build, the kotlin_library can't find any of the Android SDK classes:

aaronstacy∂oɹdʞooqɔɐɯ-ʎɔɐʇsuoɹɐɐ ☠ ~/Downloads/test  10:56:05 May23
§ bazel build --strategy=KotlinCompile=standalone :app
INFO: Found 1 target...
ERROR: /Users/aaronstacy/Downloads/test/BUILD:13:1: null failed: kotlinc failed: error executing command bazel-out/host/bin/external/com_github_jetbrains_kotlin/kotlinc -d bazel-out/android-arm-linux-androideabi-4.9-v7a-gnu-libstdcpp-fastbuild/bin/main.jar main.kt: com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
/private/var/tmp/_bazel_aaronstacy/95464d7f40ee656ba804ea72b90dd3fa/execroot/test/main.kt:3:8: error: unresolved reference: android
import android.app.Activity
       ^
/private/var/tmp/_bazel_aaronstacy/95464d7f40ee656ba804ea72b90dd3fa/execroot/test/main.kt:4:8: error: unresolved reference: android
import android.os.Bundle
       ^
...
Target //:app failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 2.452s, Critical Path: 2.28s

If I use and android_library rule and a Java file instead, it works (see the java_main rule).

Any idea how I could get this to correctly use the Android SDK to build kotlin_library's?

Also, my targets are:

§ bazel query //...:all
//:main
//:app
//:java_main
//:main_kt

I'm getting build errors trying to access some functions from third-party jars. I'm not sure if this is directly related to rules_kotlin or a more general problem with how bazel handles Kotlin code in jars. I haven't tried importing Kotlin code into Java (with bazel).

I've made a small example gist: https://gist.github.com/jmmk/2aa367c00548dd34e0a8637462493bd2. I tried all 3 of java_deps, jars, and deps as I'm not 100% clear on the difference between these. Using java_deps gets the farthest in the build, but still fails. Note that I'm using rules_maven for the dependencies, but experienced the same issue with bazel-deps.

Example build output of bazel build main_deploy.jar:

ERROR: /Users/jmmk/Desktop/bazel-kotlin/BUILD:3:1: Compiling 1 Kotlin source files to main_kt.jar failed (Exit 1).
/private/var/tmp/_bazel_jmmk/b4a1d5dca7a3dfe59337df9230eb1872/bazel-sandbox/5737601483947724481/execroot/__main__/App.kt:4:18: error: unresolved reference: launch
import tornadofx.launch
                 ^
/private/var/tmp/_bazel_jmmk/b4a1d5dca7a3dfe59337df9230eb1872/bazel-sandbox/5737601483947724481/execroot/__main__/App.kt:11:33: error: unresolved reference: launch
fun main(args: Array<String>) = launch<MyApp>(args)
                                ^
Target //:main_deploy.jar failed to build

Another library I've had fail is https://github.com/FasterXML/jackson-module-kotlin/blob/master/src/main/kotlin/com/fasterxml/jackson/module/kotlin/Extensions.kt#L15, where I got unresolved reference: jacksonObjectMapper().

Trying to run the example and getting this:

~/Development/rules_kotlin master= bazel run examples/helloworld:main_kt --verbose_failures
INFO: Found 1 target...
ERROR: /Users/myusername/Development/rules_kotlin/examples/helloworld/BUILD:8:1: null failed: kotlinc failed: error executing command 
  (exec env - \
    KOTLIN_HOME=bazel-out/host/bin/external/com_github_jetbrains_kotlin/../../../../../external/com_github_jetbrains_kotlin \
  bazel-out/host/bin/external/com_github_jetbrains_kotlin/kotlinc -d bazel-out/local-fastbuild/bin/examples/helloworld/main_kt_kt.jar -cp bazel-out/local-fastbuild/bin/examples/helloworld/libmilk.jar examples/helloworld/main.kt)

Use --sandbox_debug to see verbose messages from the sandbox.
Error: Could not find or load main class org.jetbrains.kotlin.preloading.Preloader
Use --strategy=KotlinCompile=standalone to disable sandboxing for the failing actions.
Target //examples/helloworld:main_kt failed to build
INFO: Elapsed time: 0.276s, Critical Path: 0.17s
ERROR: Build failed. Not running target.

bazel release 0.4.4-homebrew, fwiw.

Issue: When using the kotlin_binary rule to create an uberjar, one expects that direct and transitive dependencies are included. It appears that direct kotlin classes are included, but transitive kotlin classes that are transitive via a kotlin class are not. However, kotlin transitive dependencies that are transitive via a java class are included.

Disclaimer: I am new to both bazel and kotlin so likely this is user error.

Reproduce: Using a modification of rules_kotlin examples see trevorsummerssmith@2b9cb1b

We have a new Foo kotlin class. Rules (kotlin class) is modified to depend upon Foo. Main should now transitively depend upon foo.

bazel build examples/helloworld:main_kt_deploy.jar
# we expect the following to run but it does not
> java -jar bazel-bin/examples/helloworld/main_kt_deploy.jar
Exception in thread "main" java.lang.NoClassDefFoundError: examples/helloworld/Foo
	at examples.helloworld.KotlinBinaryRule.<init>(rules.kt:14)
	at examples.helloworld.MainKt.main(main.kt:11)
Caused by: java.lang.ClassNotFoundException: examples.helloworld.Foo
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:185)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:496)
	... 2 more

# Do a local modification to make Milk (a java target) depend upon Foo
# and now it works!
> java -jar bazel-bin/examples/helloworld/main_kt_deploy.jar
Foo(name=foo)
I am Kotlin! ......
... But what is soy milk?
What if soy milk is just regular milk introducing itself in Spanish?

I am happy to do some more digging here if someone can point me in the right direction.

Hypothesis (this could be totally wrong): I saw #47 and locally changed my rules.bzl to use transitive_compile_time_jars instead of full_compile_jars and then built the examples with --nouse_ijars, thinking that this would give the full transitive dependencies. However I saw no change in behavior.

From observation compiling a target which has one changed kt target in it generally takes 3-4 seconds on my mac pro. The targets are currently small -- 4-10 kt files. The Kotlin compiler is known to be 20-35% slower so it stands to reason that a background worker should make the Kotlin compiles for simple targets immaterial.

@pcj I have no idea on how to do this, but I bet you have thought about it ;), If you could you outline a design for this maybe I can do some work on this ? I have some free time at the moment.

thoughts:

• Is there some infrastructure from Jetbrains for this -- i.e., in the preloader, or could the shell be used as a background worker ? If not would nailgun make sense ?
• What would it take to tie the lifecycle of the background worker to Bazel --i.e., shutting down bazel should shutdown the worker.

The Skylark aspect rules aspect expect a java_provider set in the target output or a language specific attr -- Scala rules have the scala attribute. The kotlin_compile rule already adds a "kt" attribute -- although it is not compliant -- it misses "outputs" and other attributes. I think the future for this kind of thing is to use the java_provider and the language attributes is a interim solution.

see the aspect impl here

I don't know if the aspect logic is capable of processing macros -- but I doubt It.

kotlin_binary, kotlin_library and kotlin_test should probably be turned into rules.

When using the Kotlin rules with Bazel 0.6, I get this error:

ERROR: /tmp/bzl/BUILD:6:1: in kotlin_compile rule //:Hello_kt: 
Traceback (most recent call last):
	File "/tmp/bzl/BUILD", line 6
		kotlin_compile(name = 'Hello_kt')
	File "/usr/local/google/home/laurentlb/.cache/bazel/_bazel_laurentlb/1cf9e271f53a4fcee3da31675bf7c3a0/external/org_pubref_rules_kotlin/kotlin/rules.bzl", line 89, in _kotlin_compile_impl
		struct(files = set([kt_jar]), runfiles = ...), ...)))
	File "/usr/local/google/home/laurentlb/.cache/bazel/_bazel_laurentlb/1cf9e271f53a4fcee3da31675bf7c3a0/external/org_pubref_rules_kotlin/kotlin/rules.bzl", line 90, in struct
		set([kt_jar])
The `set` constructor for depsets is deprecated and will be removed. Please use the `depset` constructor instead. You can temporarily enable the deprecated `set` constructor by passing the flag --incompatible_disallow_set_constructor=false

Workaround is simple (specify --incompatible_disallow_set_constructor=false), but it would be nice to update the code. Ideally, it would build even with --all_incompatible_changes. See https://docs.bazel.build/versions/master/skylark/backward-compatibility.html for background.

Let me know if you need help.

Our workspace has most of the non dagger, non Kotlin transitive chain already defined. Calling the Kotlin_repositories macro breaks our build and adding omit params does not help. The external repositories defined by this library do not contain //jar packages.