bazelbuild / rules_android Goto Github PK
View Code? Open in Web Editor NEWAndroid rules for Bazel
License: Apache License 2.0
Android rules for Bazel
License: Apache License 2.0
Currently, the postsubmit configuration does not have any test_targets
.
https://github.com/bazelbuild/rules_android/blob/master/.bazelci/postsubmit.yml
At this point, the rules_android repo is 8 months stale, and still is a simple pass-through to the native rules. This is becoming a gating factor for migration, as lack of mac compability in instrumentation testing, inability to modify rules for local conditions, lack of an aar publishing rule (that can make a public, fat, shaded .aar for distribution) all of these are making it hard to move forward with bazel migration.
Please clarify some plans here, throw in a roadmap document that is current, file issues/projects for the work to be done, or in some way communicate to the users.
ACL logic does not support external packages which forces bazel users to either default everything to true/false. This is especially broken for aar_import
rules which are most likely used through maven as external repos.
See
Line 293 in e364356
Machine setup:
macOS Monterey 12.4
Android Studio Chipmunk | 2021.2.1 Patch 1
Build #AI-212.5712.43.2112.8609683, built on May 18, 2022
Runtime version: 11.0.12+0-b1504.28-7817840 x86_64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
bazel version installed: 5.0.2
I'm getting familiarized with Bazel, and trying to sync the small Android project in AS.
Tutorial: https://bazel.build/tutorials/android-app
However, when I sync via AS UI, I get the following error:
Error:(48, 19) in cc_toolchain_suite rule @local_config_cc//:toolchain: cc_toolchain_suite '@local_config_cc//:toolchain' does not contain a toolchain for cpu 'x86_64'
I have no issue building and deploying the app via the terminal.
Can someone please explain what I'm doing wrong? Thanks.
While testing the pre-alpha branch we noticed that jdk17 is being used despite us setting jdk11 as the default JDK to use. I believe this is happening because pre-alpha defines it's own custom default_java_toolchain
that gets selected instead of the one that we provide.
When passing --toolchain_resolution_debug='@bazel_tools//tools/jdk:current_java_toolchain'
we see that our default_java_toolchain
is selected:
(12:51:30) INFO: ToolchainResolution: Target platform @local_config_platform//:host: Selected execution platform @local_config_platform//:host, type @bazel_tools//tools/jdk:toolchain_type -> toolchain //:java11_for8_toolchain
But in the build failure for an Android target it's using JDK17
(12:51:46) ERROR: ...id:lib) external/remotejdk17_macos_aarch64/bin/java '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' ...
Currently fails with this:
no such target '//external:databinding_annotation_processor': target 'databinding_annotation_processor' not declared in package 'external' defined by <workspace path>/WORKSPACE (Tip: use `query "//external:*"` to see all the targets in that package) and referenced by '@bazel_tools//tools/android:databinding_annotation_processor'
Discovered while testing bazelbuild/examples#287.
When building examples/android/ndk with rules_android master branch, the build succeeds. The ManifestMerger
action has the --mergeeManfests
argument set to a list of manifest files, which includes @maven//\:androidx_appcompat_appcompat,bazel-out/android-arm64-v8a-fastbuild-ST-0b94fd089017/bin/external/maven/androidx_appcompat_appcompat_resources_processed_manifest/AndroidManifest.xml
When building with the pre-alpha branch, the ManifestMerger
does not set the flag, and the merge action fails (presumably due to the lack of needed dependencies).
Kotlin Multiplatform is now in beta and it is headed to become stable soon. Several libraries have already adopted KMP and have been publishing artifacts that make it difficult to consume Android variant of them from Bazel when using rules_android
. This issue is a request to add proper support for consuming such artifacts.
This has come up recently due to Jetpack Compose artifacts moving to KMP publishing and that breaks Bazel users because POM files do not default to the Android platform. If rules_android read Gradle module metadata, they should be able to resolve the correct artifacts.
We did another round of testing against the data bindings patches with the alpha branch of rules_android. We got further but hit a new failure case where custom styles that references Android resource attributes provided by 3rd party dependencies aren't being found during static linking.
Tested using rules_jvm_external
and bazel
HEAD.
Original report: bazelbuild/bazel#11497 (comment)
Exception in thread "main" java.lang.RuntimeException: Error during Statically linking com.google.devtools.build.android.aapt2.CompiledResources@35aea049:
Command: bazel-out/host/bin/external/androidsdk/aapt2_binary\
link\
--static-lib\
--manifest\
/var/folders/ts/25b588x11_n7f1k_2361ybr80000gp/T/android_resources_tmp17478694338343542457/manifest-aapt-dummy/AndroidManifest.xml\
--no-static-lib-packages\
--no-version-vectors\
-R\
/var/folders/ts/25b588x11_n7f1k_2361ybr80000gp/T/android_resources_tmp17478694338343542457/filtered/bazel-out/darwin-fastbuild/bin/_migrated/lib_symbols/symbols.zip\
-I\
external/androidsdk/platforms/android-30/android.jar\
--auto-add-overlay\
-o\
/var/folders/ts/25b588x11_n7f1k_2361ybr80000gp/T/android_resources_tmp17478694338343542457/lib.apk\
--java\
/var/folders/ts/25b588x11_n7f1k_2361ybr80000gp/T/android_resources_tmp17478694338343542457/java\
--output-text-symbols\
/var/folders/ts/25b588x11_n7f1k_2361ybr80000gp/T/android_resources_tmp17478694338343542457/R.txt
Output:
res/values/styles.xml:5: error: style attribute 'attr/autoSizeMinTextSize (aka com.demo.app:attr/autoSizeMinTextSize)' not found.
error: failed linking references.
at com.google.devtools.build.android.CommandHelper.execute(CommandHelper.java:42)
at com.google.devtools.build.android.AaptCommandBuilder.execute(AaptCommandBuilder.java:297)
at com.google.devtools.build.android.aapt2.ResourceLinker.linkStatically(ResourceLinker.java:272)
at com.google.devtools.build.android.ValidateAndLinkResourcesAction.main(ValidateAndLinkResourcesAction.java:207)
at com.google.devtools.build.android.ResourceProcessorBusyBox$Tool$7.call(ResourceProcessorBusyBox.java:105)
at com.google.devtools.build.android.ResourceProcessorBusyBox.processRequest(ResourceProcessorBusyBox.java:234)
at com.google.devtools.build.android.ResourceProcessorBusyBox.main(ResourceProcessorBusyBox.java:177)
Target //:lib failed to build
BUILD
load("@rules_android//rules:rules.bzl", "android_library")
android_library(
name = "lib",
custom_package = "com.demo.app",
# ... manifest, visibility, ...
resource_files = glob(["res/**/*.xml"]),
deps = [
"@maven//:androidx_appcompat_appcompat",
"@maven//:androidx_appcompat_appcompat_resources",
],
)
res/values/styles.xml
<?xml version="1.0" encoding="utf-8"?>
<resources>
<style name="FooTextAppearance">
<item name="autoSizeMinTextSize">1sp</item>
</style>
</resources>
I want use Realm in my android project. In gradle Realm requires using annotation and some classes. You can see this in link, section "Manual setup"
https://realm.io/docs/java/latest/#how-do-i-customize-dependecies-defined-by-the-realm-gradle-plugin
import io.realm.transformer.RealmTransformer
android.registerTransform(new RealmTransformer(project))
dependencies {
api "io.realm:realm-annotations:$realm_version"
api "io.realm:realm-android-library:$realm_version"
api "io.realm:realm-android-kotlin-extensions:$realm_version"
kapt "io.realm:realm-annotations-processor:$realm_version"
}
In gradle realm has import of class and annotation "kapt".
I dont know how to use annotation in Bazel. Dagger generate classes such realm, but doesn't have annotation in bazel config file. Dagger just has http_archive.
https://github.com/google/dagger#bazel
How to use Realm in Bazel ?
I don't know if this a problem with the android rules, with bazel or maybe I'm just not understanding something about how do aspects or implicit output targets work.
I'm trying to use aspects on a rule that depends on an android_library
implicit output target: name.aar. This is the only way I know to generate an aar
file but my aspect is not getting called.
I tried using the example aspect from the documentation and it is behaving the same as mine: it never gets called. Having the following code
android_library(
name = 'lib_name',
srcs = glob(['*.java']),
manifest = 'AndroidManifest.xml',
)
file_count_rule(
name = 'count',
deps = [':lib_name']
) # => WORKS FINE
file_count_rule(
name = 'count_aar',
deps = [':lib_name.aar']
) # => FAILS
If I run bazel build //:count
I get the number of files. But when I call bazel build //:count_aar
I get the following error:
<output file target lib_name.aar> doesn't contain declared provider 'FileCountInfo'
I don't really understand what implicit output targets are, or how are they treated. Is this an expected behaviour?
https://buildkite.com/bazel/rules-android/builds?page=20
This project is red on Bazel CI since Aug 19, 2019.
Is there any plan to fix it?
(00:17:49) ERROR: /workdir/android/toolchains/emulator/BUILD:15:1: no such package '@androidsdk//': The repository '@androidsdk' could not be resolved and referenced by '//android/toolchains/emulator:emulator_default'
(00:17:49) ERROR: /workdir/android/toolchains/emulator/BUILD:15:1: no such package '@androidsdk//': The repository '@androidsdk' could not be resolved and referenced by '//android/toolchains/emulator:emulator_default'
Having this error during sync in IDE
Error:/private/var/tmp/_bazel_stepango/46765f2287fb7d7cb567522e1d5568a9/external/androidsdk/BUILD.bazel:8:1: in @intellij_aspect//:intellij_info_bundled.bzl%intellij_info_aspect aspect on android_sdk rule @androidsdk//:sdk-29:
No idea what this error message means and how to fix it.
Anyway I could workaround this?
Unless projects define a Google3 style projects structure projects are required to specify the custom_package
attribute on android_library
. This forces a ManifestMerger
action on every library which is wasteful (our custom_package
values are the same as the package
we specify in every AndroidManifest.xml
.
We should introduce a flag to allow projects to use the package
attribute from AndroidManifest.xml
if desired to avoid this unnecessary step for every library.
I'm trying to build a small app using Bazel,rules_kotlin and rules_jvm_external, depending on androidx.lifecycle:lifecycle-extensions
(through maven_install
).
This library comes with a manifest declaring a provider, which is using ${applicationId}
as a placeholder - which should be replaced at merging time by the consuming application id.
I end up with this in the merged manifest:
<provider
android:name="androidx.lifecycle.ProcessLifecycleOwnerInitializer"
android:authorities="androidx.lifecycle.process.lifecycle-process"
android:exported="false"
android:multiprocess="true" />
where as with Gradle, the authority is (as expected) replaced by com.example.myapp.lifecycle-process
. The fact that it isn't replaced results in a failure at installation time, as the authority is already present on the device. Everything else looks fine in the merged manifest. Am I missing something, or is this a bug in the manifest merging process?
We are using javacopt to disabled errorprone on generated files.
If specified in .bazelrc, the corresponding javacopt is ignored (EP is showing legit errors) :
build --javacopt=-XepExcludedPaths:.*/.*genfiles/.*
If specified in the android_library rule itself, javacopt is passed properly (EP is disabled and no errors) :
java_library( name = "src_main", srcs = glob(["src/main/java/**/*.java"]), javacopts = ["-XepExcludedPaths:.*/.*genfiles/.*"], visibility = ["//visibility:public"], ... )
When the android_binary
rule generates the apk artifact, additional rules need to be added to enable codesigning.
Supported in AGP 4.2, compiles raw XML resources to intermediate flat files.
https://developer.android.com/studio/preview/features/#resource-compiler
"""potentially improving build performance, especially on Windows machines."""
Source code (entry point) is here: https://android.googlesource.com/platform/tools/base/+/studio-master-dev/build-system/aaptcompiler/src/main/java/com/android/aaptcompiler/ResourceCompiler.kt#61
Latest maven artifact: com.android.tools.build:aaptcompiler:4.2.0-alpha07
The API takes a File object. Interesting to see if it could be adapted to read resources also directly from zip files (ZipEntry) avoiding need to explode aar files for resource compilation.
No bzl_library
targets defined, making downstream repositories documenting very tricky to use rules_android
macros.
We're getting a failure in our CI because Github Actions is getting a different tarball shasum for this repo, this is a known issue that those shas can change over time: Homebrew/homebrew-core#18044
Please upload a stable release asset for these rules
Please add an example android application to this repo (android_binary). There is no guidance whatsoever on how to use these rules, how to fetch android dependencies, etc...
The last commit I can see is around 15 months ago. Is the road map that was previously laid out, on track? We still have missing features such as AAB support.
The Starlark rules are causing action bloat as a result of the transitioned outputs for each fat_apk_cpu
for ValidateAAR
and AarFileExtractor
being included in the final build.
The same amount of actions are being created between the native and Starlark rules:
# Native impl:
❯ ./bazelw aquery "mnemonic('AarFileExtractor', deps(//apps/foo:bin))" | grep "Mnemonic: AarFileExtractor" | wc -l
744
# Starlark impl:
➜ ./bazelw aquery "mnemonic('AarFileExtractor', deps(//apps/foo:bin))" | grep "Mnemonic: AarFileExtractor" | wc -l
744
However the final executed actions differ where the Starlark implementation seems to be extracting and validating each AndroidManifest.xml
for the provided --fat_apk_cpu=armeabi-v7a,arm64-v8a,x86,x86_64
architecture:
# Native impl:
➜ cat bazel_last_execution_log.json | grep '"mnemonic": "' | sort | uniq -c | sort -n
182 "mnemonic": "AarEmbeddedJarsExtractor",
182 "mnemonic": "AarEmbeddedProguardExtractor",
182 "mnemonic": "AarFileExtractor",
182 "mnemonic": "AarJarsMerger",
182 "mnemonic": "AarResourcesExtractor",
728 "mnemonic": "AarNativeLibsFilter",
# Starlark impl:
➜ cat bazel_last_execution_log.json | grep '"mnemonic": "' | sort | uniq -c | sort -n
182 "mnemonic": "AarEmbeddedJarsExtractor",
182 "mnemonic": "AarEmbeddedProguardExtractor",
182 "mnemonic": "AarJarsMerger",
182 "mnemonic": "AarResourcesExtractor",
728 "mnemonic": "AarFileExtractor",
728 "mnemonic": "AarNativeLibsFilter",
728 "mnemonic": "ValidateAAR",
We need a pure-Starlark version of android_sdk_repository
in order to decouple upcoming Android platforms changes from bazel releases.
This also helps address bazelbuild/bazel#18285.
For our build we have ~1700 Desugar
actions, it would be great if we could use D8's built in desugaring functionality.
D8Compat
which explicitly disables D8 desugaring.*_resource.jar
which do not require desugaring.While benchmarking Bazel build times against auto-generated large projects, we ran into this error in Bazel android rules that occur during resources merging. Note that Gradle (and Buck) handle these projects resources properly. Confirmed issue occurs on latest Bazel version too (3.7.2)
Repro:
Result:
Exception in thread "main" org.objectweb.asm.MethodTooLargeException: Method too large: com/rootModule/R$id. ()V
at org.objectweb.asm.MethodWriter.computeMethodInfoSize(MethodWriter.java:2089)
at org.objectweb.asm.ClassWriter.toByteArray(ClassWriter.java:458)
at com.google.devtools.build.android.resources.RClassGenerator.writeInnerClass(RClassGenerator.java:157)
at com.google.devtools.build.android.resources.RClassGenerator.writeClasses(RClassGenerator.java:123)
at com.google.devtools.build.android.resources.RClassGenerator.write(RClassGenerator.java:75)
at com.google.devtools.build.android.AndroidResourceClassWriter.writeAsClass(AndroidResourceClassWriter.java:109)
at com.google.devtools.build.android.AndroidResourceClassWriter.flush(AndroidResourceClassWriter.java:93)
at com.google.devtools.build.android.UnwrittenMergedAndroidData.writeResourceClass(UnwrittenMergedAndroidData.java:91)
at com.google.devtools.build.android.AndroidResourceMerger.mergeDataAndWrite(AndroidResourceMerger.java:197)
at com.google.devtools.build.android.AndroidResourceMerger.mergeDataAndWrite(AndroidResourceMerger.java:142)
at com.google.devtools.build.android.AndroidResourceMergingAction.main(AndroidResourceMergingAction.java:257)
at com.google.devtools.build.android.ResourceProcessorBusyBox$Tool$6.call(ResourceProcessorBusyBox.java:96)
at com.google.devtools.build.android.ResourceProcessorBusyBox.processRequest(ResourceProcessorBusyBox.java:243)
at com.google.devtools.build.android.ResourceProcessorBusyBox.runPersistentWorker(ResourceProcessorBusyBox.java:217)
at com.google.devtools.build.android.ResourceProcessorBusyBox.main(ResourceProcessorBusyBox.java:202)
AndroidResourceLink actions don't seem to be affected by APK splits.
if I have
android_binary(
name = "foo",
densities = ["hdpi"],
resource_configuration_filters = ["en"],
deps = [:library_with_resources])
android_library(
name = "library_with_resources"
resource_file = [...])
I'd expect the AndroidResourceLink
action to honor the filters applied in the binary and produce more minimal outputs but with or without those restrictions the library.ap_ outputs are the same.
Is this by design for some reason I'm missing or is this a good optimization case?
Background: A large amount of our downloads come from large intermediate outputs from AndroidResourceLink.
I suspect #10 would help with this too.
get_feature_module_paths() is broken due to the usage of Label()
. Creating a Label in an external repo will append @external_repo_name//
to the label which is not correct.
Starting with r17, Android no longer supports armeabi and instead compiles for armeabi-v7a. However, the toolchain is still registered as compatible with @platforms//cpu:arm and not @platforms//cpu:armv7. This results in toolchain not being found when targetting @platforms//cpu:armv7.
With --toolchain_resolution_debug, I see the following messages:
Type @bazel_tools//tools/cpp:toolchain_type: target platform //platforms:android_armv7: Rejected toolchain @androidndk//:arm-linux-androideabi-clang9.0.8-v7a-libcpp; mismatching values: arm
Type @bazel_tools//tools/cpp:toolchain_type: target platform //platforms:android_armv7: Rejected toolchain @local_config_cc//:cc-compiler-armeabi-v7a; mismatching values: arm
If possible, please register the toolchain as compatible with both @platforms//cpu:arm and @platforms//cpu:armv7, or provide a way to manually register the toolchain (in a generic way).
Didn't find any direct info to this.
If support, is there any guideline?
If not, is it on schedule?
I found a Bazel compile fail with Proguard 5.3.3, it would be nice if someone would share some suggestions.
https://stackoverflow.com/questions/65182300/bazel-build-android-demo-fail-with-proguard
Thank you
The bazel configurability team has a goal of making bazel build //my:android_app --platforms=//some/platform
work for all android projects without further android-specific flags needing to be set.
Part of this effort is defining android-specific platforms that can be toggled using the --platforms
flag. These platforms should live in this repository (bazelbuild/rules_android
).
Bazel version: v2.0.0
rules_android_version: v1.0.0(specific commit 7a876c3)
using my own github_archive starlark api. but bazel can't extract .tar.gz correctly.
R classes produced by android_library
are not used in the final binary, rather they are just used for Java compilation. As such it is wasteful to spend time desugaring them and this should be avoided.
Note - they shouldn't be dexed either since final R values are generated by android_binary
Hi, I hope this is the right place for this. I'm getting the following build error when I try to build an android_binary
with an android_library
that's trying to link a rust_library
INFO: Invocation ID: 0757cbc6-cf68-4e12-8afe-960a87bfcb01
ERROR: /private/var/tmp/_bazel_user/97e9600557997318199814824b024c95/external/androidndk/BUILD.bazel:41:19: in cc_toolchain_suite rule @androidndk//:toolchain-libcpp: cc_toolchain_suite '@androidndk//:toolchain-libcpp' does not contain a toolchain for cpu 'darwin'
WARNING: errors encountered while analyzing target '//:app': it will not be built
INFO: Analysis succeeded for only 2 of 3 top-level targets
INFO: Analyzed 3 targets (0 packages loaded, 0 targets configured).
INFO: Found 2 targets...
ERROR: command succeeded, but not all targets were analyzed
INFO: Elapsed time: 3.862s, Critical Path: 0.00s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
I'm trying to write a "Hello, World" app where the string "Hello, World" comes form a Rust library. Is there something I'm doing wrong here?
The zip below contains a repro case. I've also added my BAZEL.build
and WORKSPACE
files here for convenience.
repro.zip
BUILD.bazel
load("@io_bazel_rules_rust//rust:rust.bzl", "rust_library")
load("@build_bazel_rules_android//android:rules.bzl", "android_binary", "android_library")
package(default_visibility = ["//visibility:public"])
rust_library(
name = "repro",
srcs = glob(include = ["src/**/*.rs"]),
crate_type = "cdylib",
deps = [
"//cargo:ferris_says",
],
tags = ["manual"],
)
android_library(
name = "lib",
srcs = glob(include = ["android/src/main/**/*.java"]),
custom_package = "com.repro.example",
manifest = "android/src/main/LibraryManifest.xml",
resource_files = glob(["android/src/main/res/**/*"]),
deps = [
"//:repro",
"@maven//:androidx_appcompat_appcompat",
"@maven//:androidx_constraintlayout_constraintlayout",
"@maven//:androidx_navigation_navigation_fragment",
"@maven//:androidx_navigation_navigation_ui",
"@maven//:com_google_android_material_material",
],
tags = ["manual"],
)
android_binary(
name = "app",
custom_package = "com.repro.example",
manifest = "src/main/AndroidManifest.xml",
manifest_values = {
"minSdkVersion": "28",
"targetSdkVersion": "28",
},
deps = [":lib"],
)
WORKSPACE
workspace(name = "repro")
load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
git_repository(
name = "bazel_skylib",
commit = "d35e8d7bc6ad7a3a53e9a1d2ec8d3a904cc54ff7",
remote = "https://github.com/bazelbuild/bazel-skylib.git",
shallow_since = "1593183852 +0200",
)
load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")
bazel_skylib_workspace()
git_repository(
name = "io_bazel_rules_rust",
commit = "f41a008d3e36e70b9deba9e4ed6cd914e8f36a49",
remote = "https://github.com/bazelbuild/rules_rust.git",
shallow_since = "1593798220 +0200",
)
load("@io_bazel_rules_rust//:workspace.bzl", "bazel_version")
bazel_version(name = "bazel_version")
load("@io_bazel_rules_rust//rust:repositories.bzl", "rust_repositories")
rust_repositories(version = "1.45.0", edition = "2018")
load("//cargo:crates.bzl", "raze_fetch_remote_crates")
raze_fetch_remote_crates()
git_repository(
name = "build_bazel_rules_android",
remote = "https://github.com/bazelbuild/rules_android.git",
commit = "e8fbc49f913101e846235b9c9a31b3aa9788364a",
shallow_since = "1593973351 -0700",
)
git_repository(
name = "rules_jvm_external",
remote = "https://github.com/bazelbuild/rules_jvm_external.git",
commit = "a8300d874fcda9b3849c288f4ad57c53a454953e",
shallow_since = "1594195655 +0800",
)
android_sdk_repository(
name = "androidsdk",
api_level = 28,
build_tools_version = "28.0.3",
)
android_ndk_repository(
name = "androidndk",
api_level = 28,
)
load("@rules_jvm_external//:defs.bzl", "maven_install")
maven_install(
artifacts = [
"androidx.appcompat:appcompat:1.1.0",
"com.google.android.material:material:1.0.0",
"androidx.constraintlayout:constraintlayout:1.1.3",
"androidx.navigation:navigation-fragment:2.1.0",
"androidx.navigation:navigation-ui:2.1.0",
],
repositories = [
"https://jcenter.bintray.com/",
"https://maven.google.com",
],
)
We're getting some of these errors:
WARNING: Download from https://github.com/bazelbuild/rules_android/archive/v0.1.1.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException Checksum was 7cb2b2e11aafe08b2ccb6eeb11f21f52c45e1ad1cdd0132ddf7bde8462bd9e20 but wanted cd06d15dd8bb59926e4d65f9003bfc20f9da4b2519985c27e190cddc8b7a7806
Indeed verifying the sha for me on my macbook shows it has changed:
$ curl -sL https://github.com/bazelbuild/rules_android/archive/v0.1.1.zip | shasum -a 256 -b
cd06d15dd8bb59926e4d65f9003bfc20f9da4b2519985c27e190cddc8b7a7806 *-
Hi @jin,
We are working (@gonzaloruizdevilla) on a project, where the use of databinding functionality is needed, but we encounter the following problem:
build did NOT complete successfully (33 packages loaded, 1068 targets configured)
Internal error thrown during build. Printing stack trace: java.lang.RuntimeException: Unrecoverable error while evaluating node '//Android/Components/Mobisec-STGAuthenticators-Android/src/main:authenticatorslib BuildConfigurationValue.Key[f3f502b3f0d1aee893a3fe534606da2c] false' (requested by nodes )
at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:528)
at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:399)
at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)
Caused by: java.lang.NullPointerException
at com.google.devtools.build.lib.rules.java.JavaCompilationHelper.usesAnnotationProcessing(JavaCompilationHelper.java:433)
at com.google.devtools.build.lib.rules.android.AndroidCommon.initJava(AndroidCommon.java:613)
at com.google.devtools.build.lib.rules.android.AndroidCommon.init(AndroidCommon.java:547)
at com.google.devtools.build.lib.rules.android.AndroidLibrary.create(AndroidLibrary.java:201)
at com.google.devtools.build.lib.rules.android.AndroidLibrary.create(AndroidLibrary.java:43)
at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createRule(ConfiguredTargetFactory.java:331)
at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createConfiguredTarget(ConfiguredTargetFactory.java:182)
at com.google.devtools.build.lib.skyframe.SkyframeBuildView.createConfiguredTarget(SkyframeBuildView.java:767)
at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.createConfiguredTarget(ConfiguredTargetFunction.java:891)
at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.compute(ConfiguredTargetFunction.java:351)
at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:451)
Please, could you tell me what the problem is due to.
Are we doing something wrong?
The Build file we use is the following:
#############################################
package(default_visibility = ["//visibility:public"])
load("@rules_jvm_external//:defs.bzl", "artifact")
load("@rules_android//android:rules.bzl", "android_library")
MANIFEST = "AndroidManifest.xml"
PACKAGE = "com.globile.santander.mobisec.authenticators"
android_library(
name = "authenticatorslib",
manifest = MANIFEST,
custom_package = PACKAGE,
enable_data_binding = True,
resource_files = glob(["res/"]),
srcs = glob(["java//*.java"]),
deps = [
"//Android/Components/Mobisec-SCAL-Android/src/main:scallib",
artifact("androidx.appcompat:appcompat"),
artifact("androidx.fragment:fragment"),
artifact("androidx.core:core"),
artifact("androidx.lifecycle:lifecycle-runtime"),
artifact("androidx.lifecycle:lifecycle-viewmodel"),
artifact("androidx.lifecycle:lifecycle-common"),
artifact("androidx.lifecycle:lifecycle-livedata"),
artifact("androidx.lifecycle:lifecycle-extensions"),
artifact("androidx.lifecycle:lifecycle-livedata-core"),
artifact("androidx.drawerlayout:drawerlayout"),
artifact("androidx.annotation:annotation"),
artifact("com.crashlytics.sdk.android:crashlytics"),
artifact("io.fabric.sdk.android:fabric"),
artifact("com.google.code.gson:gson"),
artifact("io.reactivex.rxjava2:rxjava"),
artifact("io.reactivex.rxjava2:rxandroid"),
artifact("org.reactivestreams:reactive-streams"),
artifact("com.journeyapps:zxing-android-embedded"),
artifact("androidx.savedstate:savedstate"),
artifact("com.squareup.okhttp3:okhttp"),
artifact("com.squareup.okhttp3:logging-interceptor"),
artifact("net.zetetic:android-database-sqlcipher"),
artifact("com.google.android.gms:play-services-vision"),
artifact("com.squareup.retrofit2:converter-gson"),
artifact("com.squareup.retrofit2:retrofit"),
artifact("com.squareup.retrofit2:converter-scalars"),
artifact("com.android.support:support-annotations"),
artifact("com.android.support:design"),
artifact("android.arch.lifecycle:extensions"),
artifact("com.android.support:cardview-v7"),
artifact("com.android.support:appcompat-v7"),
artifact("com.android.support.constraint:constraint-layout"),
artifact("com.android.databinding:baseLibrary"),
artifact("com.android.databinding:adapters"),
]
)
######################################
Best Regards.
Running bazel mobile-install //app:app_binary --start_app
fails when installing an incremental apk on a device running Android 11. This only happens when targetSdkVersion="30"
in my AndroidManifest.xml
.
The exact error output is
Error:
adb command: ['external/androidsdk/platform-tools/adb', 'install', '-r', './bazel-out/darwin-fastbuild/bin/app/src/app_binary_incremental.apk']
return code: 1
stdout: Performing Streamed Install
stderr: adb: failed to install ./bazel-out/darwin-fastbuild/bin/app/src/app_binary_incremental.apk: Failure [-124: Failed parse during installPackageLI: Targeting R+ (version 30 and above) requires the resources.arsc of installed APKs to be stored uncompressed and aligned on a 4-byte boundary]
I believe the error could be related to this behavior change for apps targeting Android 11. But, there might also be a case for the error being related to how incremental apks are generated.
I'm able to manually run adb install bazel-out/darwin-fastbuild/bin/app/src/app_binary.apk
without issue. However, bazel mobile-install
runs adb install -r bazel-out/darwin-fastbuild/bin/app/src/app_binary_incremental.apk
which results in a failure.
Also, running zipalign -c -v 4 bazel-out/darwin-fastbuild/bin/app/src/app_binary_incremental.apk
outputs a Verification FAILED
message.
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
## BEGIN build_bazel_rules_android ##
RULES_ANDROID_VERSION = "0.1.1"
RULES_ANDROID_SHA = "cd06d15dd8bb59926e4d65f9003bfc20f9da4b2519985c27e190cddc8b7a7806"
http_archive(
name = "build_bazel_rules_android",
sha256 = RULES_ANDROID_SHA,
strip_prefix = "rules_android-%s" % RULES_ANDROID_VERSION,
type = "zip",
urls = ["https://github.com/bazelbuild/rules_android/archive/v%s.zip" % RULES_ANDROID_VERSION],
)
load("@build_bazel_rules_android//android:rules.bzl", "android_sdk_repository")
load("@build_bazel_rules_android//android:rules.bzl", "android_ndk_repository")
android_sdk_repository(
name = "androidsdk",
api_level = 30,
build_tools_version = "30.0.1",
)
android_ndk_repository(
name = "androidndk",
api_level = 28,
)
## END build_bazel_rules_android ##
## BEGIN io_bazel_rules_kotlin ##
RULES_KOTLIN_VERSION = "legacy-1.3.0"
RULES_KOTLIN_SHA = "4fd769fb0db5d3c6240df8a9500515775101964eebdf85a3f9f0511130885fde"
http_archive(
name = "io_bazel_rules_kotlin",
sha256 = RULES_KOTLIN_SHA,
strip_prefix = "rules_kotlin-%s" % RULES_KOTLIN_VERSION,
type = "zip",
urls = ["https://github.com/bazelbuild/rules_kotlin/archive/%s.zip" % RULES_KOTLIN_VERSION],
)
load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kotlin_repositories", "kt_register_toolchains")
kotlin_repositories()
kt_register_toolchains()
## END io_bazel_rules_kotlin ##
## Begin robolectric ##
ROBOLECTRIC_VERSION = "4.4"
ROBOLECTRIC_SHA = "f217e945d531828abced835ca467481827549250d57ae1b511899901cc4aeed9"
http_archive(
name = "robolectric",
sha256 = ROBOLECTRIC_SHA,
strip_prefix = "robolectric-bazel-%s" % ROBOLECTRIC_VERSION,
type = "zip",
urls = ["https://github.com/robolectric/robolectric-bazel/archive/%s.zip" % ROBOLECTRIC_VERSION],
)
load("@robolectric//bazel:robolectric.bzl", "robolectric_repositories")
robolectric_repositories()
## End robolectric ##
## Begin rules_jvm_external ##
RULES_JVM_EXTERNAL_VERSION = "3.3"
RULES_JVM_EXTERNAL_SHA = "d85951a92c0908c80bd8551002d66cb23c3434409c814179c0ff026b53544dab"
http_archive(
name = "rules_jvm_external",
sha256 = RULES_JVM_EXTERNAL_SHA,
strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_VERSION,
type = "zip",
urls = ["https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % RULES_JVM_EXTERNAL_VERSION],
)
load("@rules_jvm_external//:defs.bzl", "maven_install")
maven_install(
artifacts = [
"androidx.appcompat:appcompat:1.2.0",
"androidx.core:core-ktx:1.3.1",
"androidx.constraintlayout:constraintlayout:2.0.1",
"androidx.test:monitor:1.2.0",
"androidx.test.espresso:espresso-core:3.3.0",
"androidx.test.ext:junit:1.1.2",
"junit:junit:4.12",
"org.robolectric:robolectric:4.4",
],
repositories = [
"https://jcenter.bintray.com",
"https://maven.google.com",
"https://repo1.maven.org/maven2",
],
)
## End rules_jvm_external ##
load("@build_bazel_rules_android//android:rules.bzl", "android_library")
load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_android_library")
load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_jvm_test")
android_binary(
name = "app_binary",
custom_package = "com.veganafro.app",
manifest = "main/AndroidManifest.xml",
shrink_resources = 0,
deps = [
"//app/src:app_lib",
],
)
kt_android_library(
name = "app_lib",
custom_package = "com.veganafro.app",
manifest = "main/AndroidManifest.xml",
resource_files = glob(["main/res/**"]),
srcs = glob(["main/java/com/veganafro/app/*.kt"]),
visibility = ["//:__pkg__"],
deps = [
"@maven//:androidx_appcompat_appcompat",
"@maven//:androidx_constraintlayout_constraintlayout",
"@maven//:androidx_core_core_ktx",
],
)
kt_jvm_test(
name = "app_unit_test",
srcs = glob(["test/java/com/veganafro/app/*.kt"]),
test_class = "com.veganafro.app.ExampleUnitTest",
deps = [
"@maven//:junit_junit",
],
)
android_local_test(
name = "app_android_test",
manifest = "androidTest/AndroidManifest.xml",
test_class = "com.veganafro.app.ExampleInstrumentedTest",
deps = [
"//app/src/androidTest/java/com/veganafro/app:app_android_test_lib",
"@maven//:org_robolectric_robolectric",
"@robolectric//bazel:android-all",
],
)
├── WORKSPACE
└── app
├── src
│ └── main
│ ├── AndroidManifest.xml
│ ├── java
│ │ └── com
│ │ └── veganafro
│ │ └── app
│ │ └── MainActivity.kt
│ └── res
│ ├── drawable
│ ├── drawable-v24
│ ├── layout
│ │ └── activity_main.xml
│ ├── mipmap-anydpi-v26
│ ├── mipmap-hdpi
│ ├── mipmap-mdpi
│ ├── mipmap-xhdpi
│ ├── mipmap-xxhdpi
│ ├── mipmap-xxxhdpi
│ └── values
│ ├── colors.xml
│ ├── strings.xml
│ └── styles.xml
├── proguard-rules.pro
└── BUILD
0.1.1
Build label: 3.7.2-homebrew
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Dec 17 23:06:38 2020 (1608246398)
Build timestamp: 1608246398
Build timestamp as int: 1608246398
x86_64-apple-darwin19
Hi wonderful Bazelers,
I'd like to float an idea for making official something I find myself repeatedly rebuilding on top of platforms when I use it with the Android rules.
Android has its own names for CPU architectures that differ slightly from the Bazel CPU names over in https://github.com/bazelbuild/platforms. In order to use libraries, including Google ones, like Firebase, or to otherwise build Android things in Starlark, you have to hardcode that mapping.
It'd be great if rules_android would could be the source of truth for that mapping at some point in the future! I know there's lots left to do before a v1 gets out, but I wanted to chime in that I think this is going to be a generally helpful feature more broadly than just being needed for the implementation of the starlark rules.
The mapping could look something like:
BAZEL_CPU_ANDROID_ARCH_PAIRS = (
("armv7", "armeabi-v7a"),
("arm64", "arm64-v8a"),
("x86_32", "x86"),
("x86_64", "x86_64"),
)
Happy to contribute of course--and would have made this a PR--except that I'm not sure quite how this repo is being managed wrt the starlark rules alpha branch.
Cheers!
Chris
(ex-Googler)
Are there plans to add CI integrations with tests and examples for the rules_android alpha branch?
mobile-install v2 is a Starlark rewrite of the native mobile-install
command in Bazel.
This is the tracking bug for the open sourcing of v2.
Given:-
android_binary("app") -> android_library("libA") -> android_library("libB) -> android_library("libc") -> "res/values/strings.xml"
bazel build //:app
Adding a new string resource in "libc" causes a rebuild of all upstream targets ("app", "libA', "libB").
We're using android_strict_resource_deps as well as the namespaced_r_classes bazelbuild/bazel#11385 I have open.
As I expect there is no resource merging or linking for "libA' + "libB", just a single AndroidResourceCompiler
for "libC" and a single AndroidAapt2
action to link the changed resource into the android_binary.
What I am surprised about is that all upstream targets have a KotlinCompile
action associated with them. This is because the new resource has changed the R.class and libC_resource.jar which is an input to all upstream targets.
This is where my knowledge of Bazel is a little hazy, but is it expected that the libC_resource.jar output is given as an input to ALL upstream targets (i.e: "app", "libA", "libB")?
I think, what we want, since we have strict_java_deps is that the change to the resource would only require recompiling the Java/Kotlin for libC + libB (since B is permitted to depend on libC.R.class as its a direct dependency)
I noticed that all transitive resource jars are collected and added to AndroidLibraryResourceClassJarProvider
here:-
Commenting out the collection of transitive resource jars and only including the locally produced resource jar has no affect though.
Of is this is correct, do we need to put some kind of ijar in place for upsteam targets that are not direct parents?
@donaldchai - would appreciate your thoughts here. I'm happy submit a PR if you think this is valid change and can give a few pointers.
Thanks, Jonathan
Now that name-spaced resources are supported, teams that exclusively use this feature have no need for resource merging work, which ends up being a substantial fraction of a clean (or low incrementality) build.
Reduce unnecessary work during an android build.
Write any trivial app which consumes .aars, and depend on a namespaced resource directly (rather than in the local R.class). Then build. All the resources are merged into the current library, even though the current library doesn't require local references to these.
Mac OS X 10.14.5
bazel info release
?release 0.28.1
Discussion with @jin has confirmed that the resource merging is (presently) mandatory.
For context, we turned this off in gradle and achieved about a 30% reduction in build times. Bazel's sensitivity to caching/incrementality will reduce this gain in high-incrementality situations, but might increase it in low-incrementality scenarios. Our observations from profiles are that we spend about a substantial fraction of our build, front-loaded, pegging our workers with this work.
Note: Copied from bazelbuild/bazel#8577 which I'm closing.
0.1.1
Build label: 3.7.1-homebrew
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Nov 24 20:21:33 2020 (1606249293)
Build timestamp: 1606249293
Build timestamp as int: 1606249293
x86_64-apple-darwin19
I'm attempting to run bazel mobile-install
for an android_binary
target. The app is the simple hello world app generated by the single activity app base in Android Studio.
This is the incorrect output I'm seeing:
$ bazel mobile-install //app/src:app_binary --start_app --incremental
INFO: Analyzed target //app/src:app_binary (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /Users/jeremymuhia/development/mono/android/app/src/BUILD:5:15: Installing //app/src:app_binary incrementally failed (Exit 1): incremental_install failed: error executing command bazel-out/darwin-py2-opt-exec-2B5CBBC6/bin/external/bazel_tools/tools/android/incremental_install --output_marker bazel-out/darwin-fastbuild/bin/app/src/app_binary_files/incremental_deploy_marker ... (remaining 10 argument(s) skipped)
Traceback (most recent call last):
File "/private/var/tmp/_bazel_jeremymuhia/f9dfbeb00953a93e3d2ab5498ee6f23d/execroot/__main__/bazel-out/darwin-py2-opt-exec-2B5CBBC6/bin/external/bazel_tools/tools/android/incremental_install.runfiles/bazel_tools/tools/android/incremental_install.py", line 25, in <module>
from concurrent import futures
ImportError: No module named concurrent
Target //app/src:app_binary failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.270s, Critical Path: 0.11s
INFO: 3 processes: 3 internal.
FAILED: Build did NOT complete successfully
This seems peculiar because when I build an apk with bazel build //app/src:app_binary
and install it manually with adb install bazel-bin/app/src/app_binary.apk
, I can see that the app is installed on the device and is without any runtime errors.
This issue describes the current state of dynamic features in Bazel, the differences between Bazel and Gradle as well as the required work to bring them to parity with Gradle to support Open Source builds. This document focuses on the two main issues with the current version of the rules: native library support and dex support. Finally, an appendix with all the issues we encountered can be seen at the end. These issues are minor and are only listed for documentation purposes.
Currently Bazel supports dynamic features via the android_feature_module
and android_application
rules. In order to create a feature module, you declare a new android_feature_module
target and pass it to an android_application
target via the feature_modules
attribute. The android_feature_module
can’t be used on its own. It must always be used through android_application
.
The android_feature_module
rule supports creating dynamic features that contain native libraries and/or assets only. There is no support for dynamic features that contain Java/Kotlin code or Android resources.
In order to specify the dependencies of a dynamic feature, one can either pass an android_library
target through the library
attribute or/and an android_binary
target through the binary
attribute of the rule. The full list of attributes the rule takes can be found here.
The assets for the dynamic feature are collected from the set of transitive dependencies of the dynamic feature (via library
and binary
attrs).
The native libraries the dynamic feature are collected from the APK of the Android Binary target passed via the binary
attr of the feature module rule. The android_application
rule extracts all the libraries in the binary apk and packages them with the module. I.e. the module will include all the libraries in the transitive dependencies of the binary.
Gradle supports dynamic features via the dynamicFeatures attribute. The base module is added as dependency of every feature module by default in the build.gradle file (not this is fundamentally not possible in Bazel due to the cyclic dependency). The module will contain the direct dependencies specified in the build.gradle. Only artifacts that are explicitly specified in the build.gradle file are considered to be part of the feature module. The dependency on the base module is only used for compilation purposes. This means that multiple feature modules can have common dependencies without ending up with multiple duplicate dependencies in different modules.
Assets, native libraries, Java/Kotlin code and resources are supported.
The current approach used by Bazel to build feature modules, while simple, has issues when building complex feature modules that include more than just assets.
The native libraries support relies on building an intermediate APK which is only used to extract the .so files. This binary can’t be reused across feature modules, since doing so would mean duplicating the native binaries included in each feature module. In addition to that, this future is implemented by inspecting the IdeInfo
provider in order to detect if the target includes native binaries, which seems like a hacky use of the provider.
The same approach could be used in order to support Dex files (create a binary target and extract the dex files from the APK) but this would also fall into duplicate dependencies being included in each feature module in any case where different feature modules share common dependencies. If we wanted to use this approach, we’d need to take care into crafting targets with a disjoint set of dependencies (which is infeasible if we want to share code) or craft custom targets marked as neverlink in order to be used by the feature module. Doing this would mean writing a lot of boiler plate targets and could prove to be very error prone.
In order to enable dynamic features with Bazel in Gradle repos with minimal modifications to the current builds working with Gradle, we propose flattening the native_feature_module
rule and allow users to explicitly specify the dependencies that need to be included (android_library and native library deps). Concretely we propose replacing the library
and binary
attributes for a deps
attribute.
The deps attribute would take kt_android_library
, android_library and aar_import targets. For dependencies with code it will just dex the code just in that dependency (it will not dex the set of transitive deps) and will collect any native library included in that dependency (no transitive libraries as well). In order to enable dexing, the code from DexArchiveAspect will need to be lifted into starlark and applied at the library level instead of the binary level (most of the code can be simplified since there is significant legacy code that can be ignored).
One remaining issue with this approach is that R8/proguarding feature modules requires the full set of binary dexes and feature module dexes. Currently the R8 code lives in the native android binary so any support for R8 with feature modules is TBD until there is more clarity on the future of the android_binary
rule.
r24 isn't supported in master due to the toolchain path changes after r21. On the 'pre-alpha' branch, this warning and error appears when using r24 NDK (and I suppose r22 and r23). Is there a workaround for supporting r24 in bazel? If not, is it reasonable to use SDK 30 with r21?
WARNING: The major revision of the Android NDK referenced by android_ndk_repository rule 'androidndk' is 24. The major revisions supported by Bazel are [10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21]. Bazel will attempt to treat the NDK as if it was r21. This may cause compilation and linkage problems. Please download a supported NDK version. ERROR: /usr/local/google/home/mchinen/scm/lyra/android_example/BUILD:26:16: While resolving toolchains for target //android_example:lyra_android_lib: no matching toolchains found for types @rules_android//toolchains/android:toolchain_type, @rules_android//toolchains/android_sdk:toolchain_type ERROR: /usr/local/google/home/mchinen/scm/lyra/android_example/BUILD:46:15: While resolving toolchains for target //android_example:lyra_android_example_RESOURCES_DO_NOT_USE: no matching toolchains found for types @rules_android//toolchains/android:toolchain_type
Feature modules are broken due to merge_feature_manifest.par missing (https://github.com/bazelbuild/rules_android/blob/pre-alpha/rules/android_application/attrs.bzl#L62)
Can this be switched to the vanilla manifest merger? Other than having some special logic to insert ${MODULE_TITLE}
in the manifest I can't tell what else it might be doing.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.