android / performance-samples Goto Github PK

View Code? Open in Web Editor NEW

1.2K 98.0 207.0 4.99 MB

Samples to show APIs and best practices in Performance on Android

Home Page: https://d.android.com/topic/performance/overview

License: Apache License 2.0

Kotlin 93.08% Shell 0.32% TypeScript 6.60%

android performance samples

performance-samples's Introduction

Android Performance

A collection of samples using the performance libraries.

You can learn more about Android Performance at https://d.android.com/performance.

Macrobenchmark sample

This sample shows how to use the Macrobenchmark library for testing application startup and runtime performance cases, such as scrolling a RecyclerView or LazyColumn to measure jank.

Baseline Profiles can be generated with the Macrobenchmark library. This sample contains code to generate a baseline profile.

The sample also includes a GitHub workflow to run Macrobenchmarks on Firebase Test Lab. For more information refer to the README.md.

The Macrobenchmark sample also demonstrates using Jetpack Compose with Macrobenchmark.

Microbenchmark sample

This sample shows how to use the Benchmark library to benchmark code and UI from library modules.

Important Notes

Make sure your device's screen is on before running benchmarks

JankStats sample

This sample shows how to set up and use the JankStats library to detect janky frames.

Reporting Issues

You can report an Issue with a sample using this repository. If you find an issue with a specific library, report it using the corresponding issue tracker link available in the sample README file.

Additional Resources

PagingWithNetworkSample

Example of benchmarking an androidx.paging backed RecyclerView

WorkManagerSample

Example of benchmarking asynchronously scheduled background work

performance-samples's People

Contributors

Stargazers

Watchers

Forkers

sairithy abrahamzn bhanditz albodelu grittycedar vincentlao vkotovv daparna66 yunuseker tuan25abc zakharovsergey1000 agmadkafaar sahebojha xiang1912 tchigher mohand1993 muxi166 jqhan pp-lucaal-sportsbet lonyzone anandita-87 manish-bannur alwin-lin xdocuments ashishsingh009 ag143 sahilmidha amitmsingh poolooloo dalalsunil1986 ameerirsh mdlglobal-atlassian-net i-eugene nghianguyen979 mrdragonforest jemmycalak dev-mgkaung thetruthmyg maihongyenit bluelucky jimmychung1991 jenoturkey przor3n kanaksony hearsilent hackr617 global-localhost kharisma-aw ar7ene vjgarciag96 yanghuadong-mobile-researcher xjbclz droidiq pbajurko tikurahul 00mjk hendrysetiadi89 nity-su isabella232 i-cat saiprasad16 jlinkuo burgess2017 dayanruben luismendes070 saldisobi anjalsaneen qingmei2 zmarkan emilywu2046 ayush1120 nyctonio imbeerus maxmood96 mozuck sumeht mlykotom venator85 ranjith068 esafirm mcomella donatienthorez xuchuwei satyarajrapido foghina daivinh11 pyricau lihou fizietechd kikoso lgd2009 chethaase cybernetics classicvalues huanhailiuxin df13954 gyogle zhangrui49 kingsmentor deepfish123

performance-samples's Issues

Benchmark for Compose

How to implement microbenchmark test for compose function?

Benchmark cold start of an application with login screen

I would like to use macro benchmark to measure the cold start of an application after the user has logged in.
How can I do that because MacroBenchmark clear app's data before each run ?

In Jank Stat Aggregator report function frame data is changing.

I have put logs on the jank stat listener its shows a janky frame is found and when I see the results in the report listener it shows the frame is not Janky. Some weird observations I have found with this library.

Update to Compose October '22

Would be great to update the Compose samples to use the Compose BOM. See android/user-interface-samples#399 for an example of how to do such a migration.

The benchmark setup with release target is unrealistic

Seems like useful sample to show both benchmark and release targets.

Benchmark Tests do not run in Android Version 11

I have been trying to get my hands on benchmarking for a long time now. It seems to work fine on physical devices with android 13 but doesn't seem to to work on android 11.

Below is the logcat screenshot :

Is there any workaround this, since, it is important to benchmark an app in different android versions.

Error: Run configuration ExampleStartupBenchmark is not supported in the current project. Cannot obtain the package.

When I click on Run action for the purpose of running the vanilla benchmark test Edit Configuration dialog appears with this error.

Fix package Name on Readme.

i think package name is com.example.macrobenchmark.target

https://github.com/android/performance-samples/blame/main/MacrobenchmarkSample/functions/README.md#L21

Macrobenchmark test sample fails

After starting tests:

Starting 5 tests on Hammer_IRON_3_LTE - 9

com.example.macrobenchmark.SmallListStartupBenchmark > startup[mode=COLD][Hammer_IRON_3_LTE - 9] FAILED 
        java.util.NoSuchElementException: Collection contains no element matching the predicate.
        at androidx.benchmark.macro.perfetto.StartupTimingQuery.findEndRenderTimeForUiFrame(StartupTimingQuery.kt:202)

com.example.macrobenchmark.TrivialStartupBenchmark > startup[mode=COLD][Hammer_IRON_3_LTE - 9] FAILED 
        java.util.NoSuchElementException: Collection contains no element matching the predicate.
        at androidx.benchmark.macro.perfetto.StartupTimingQuery.findEndRenderTimeForUiFrame(StartupTimingQuery.kt:202)

[Device]: Error during Sync: Remote object doesn't exist!

Error while pulling test data from device.
java.io.IOException: Failed to pull \storage\emulated\0\Android\data\com.example.macrobenchmark.target\files\test_data\SmallListStartupBenchmark_startup[mode=COLD]_iter000_2022-01-11-18-57-13.perfetto-trace from device
        at com.android.build.gradle.internal.testing.ConnectedDevice.pullFile(ConnectedDevice.java:212)
        at com.android.build.gradle.internal.testing.SimpleTestRunnable$3.processNewLines(SimpleTestRunnable.java:448)
        at com.android.ddmlib.MultiLineReceiver.addOutput(MultiLineReceiver.java:104)
        at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:706)
        at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:470)
        at com.android.ddmlib.internal.DeviceImpl.executeShellCommand(DeviceImpl.java:708)
        at com.android.build.gradle.internal.testing.ConnectedDevice.executeShellCommand(ConnectedDevice.java:180)
        at com.android.build.gradle.internal.testing.SimpleTestRunnable.executeShellCommand(SimpleTestRunnable.java:550)
        at com.android.build.gradle.internal.testing.SimpleTestRunnable.pullTestData(SimpleTestRunnable.java:468)
        at com.android.build.gradle.internal.testing.SimpleTestRunnable.run(SimpleTestRunnable.java:258)
        at com.android.build.gradle.internal.tasks.Workers$ProfileAwareExecutorServiceAdapter$submit$submission$1.run(Workers.kt:165)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1406)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1016)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1665)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1598)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
Caused by: com.android.ddmlib.SyncException: Remote object doesn't exist!
        at com.android.ddmlib.SyncService.pullFile(SyncService.java:341)
        at com.android.ddmlib.internal.DeviceImpl.pullFile(DeviceImpl.java:1079)
        at com.android.build.gradle.internal.testing.ConnectedDevice.pullFile(ConnectedDevice.java:205)
        ... 16 more

Build environment:

Hammer Iron 3 LTE - Android 10 - API 28
Android Studio version: Arctic Fox | 2020.3.1 Patch 2
Android Build Tools version: 30.0.2
Gradle version: 7.0.4

!!! JUnit version 3.8 or later expected:

getting this error even after all work arounds
#24 - not working

!!! JUnit version 3.8 or later expected:

java.lang.ClassNotFoundException: junit.framework.ComparisonFailure
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.intellij.rt.execution.junit.JUnitStarter.junitVersionChecks(JUnitStarter.java:221)
at com.intellij.rt.execution.junit.JUnitStarter.canWorkWithJUnitVersion(JUnitStarter.java:207)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:61)

how to allow runtime location permission during macro benchmark startup test

Macrobenchmark fails to pull trace on Android 12

Steps to reproduce issue

Clone performance-samples project
Open MacrobenchmarkSample in Android Studio. I'm using Bumblebee 2021.1.1 Beta 1.
Run TrivialStartupBenchmark on physical device with Android 12 (Pixel 5 in my case)

Expected output

Tests are passing

Actual output

Macrobenchmark fails to pull trace file

java.io.FileNotFoundException: /storage/emulated/0/Android/data/com.example.macrobenchmark/cache/TrivialStartupBenchmark_startup[mode=COLD]_iter000_2021-10-26-07-09-40.perfetto-trace: open failed: EACCES (Permission denied)
	at libcore.io.IoBridge.open(IoBridge.java:575)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:236)
	at kotlin.io.FilesKt__FileReadWriteKt.appendBytes(FileReadWrite.kt:115)
	at androidx.benchmark.perfetto.UiStateKt.appendUiState(UiState.kt:45)
	at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmark(Macrobenchmark.kt:200)
	at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmarkWithStartupMode(Macrobenchmark.kt:273)
	at androidx.benchmark.macro.junit4.MacrobenchmarkRule.measureRepeated(MacrobenchmarkRule.kt:68)
	at androidx.benchmark.macro.junit4.MacrobenchmarkRule.measureRepeated$default(MacrobenchmarkRule.kt:58)
	at com.example.macrobenchmark.StartupUtilsKt.measureStartup(StartupUtils.kt:32)
	at com.example.macrobenchmark.TrivialStartupBenchmark.startup(TrivialStartupBenchmark.kt:34)
	... 37 trimmed
Caused by: android.system.ErrnoException: open failed: EACCES (Permission denied)
	at libcore.io.Linux.open(Native Method)
	at libcore.io.ForwardingOs.open(ForwardingOs.java:567)
	at libcore.io.BlockGuardOs.open(BlockGuardOs.java:273)
	at libcore.io.ForwardingOs.open(ForwardingOs.java:567)
	at android.app.ActivityThread$AndroidOs.open(ActivityThread.java:7728)
	at libcore.io.IoBridge.open(IoBridge.java:561)
	... 47 more

It looks like the file exists cause I can pull it using adb pull

adb pull /storage/emulated/0/Android/data/com.example.macrobenchmark/cache/TrivialStartupBenchmark_startup[mode=COLD]_iter000_2021-10-26-07-09-40.perfetto-trace

/storage/emulated/0/Android/data/com.example.macrobenchmark/... file pulled, 0 skipped. 38.0 MB/s (6161528 bytes in 0.155s)

MacrobenchmarkSample doesn't run with Android Studio Electric Eel

Trying to run a macrobenchmark test from the editor.

Works with Dolphin

Not able to run project MacrobenchmarkSample

Sample pages return 404

ScrollBenchmark Fling Behavior on Large Screens

Should fling behavior work fine for TvLazyColumns / TvLazyRows on large screens?

The baseline profile install broadcast was not received. This most likely means that the profileinstaller library is missing from the target apk

I change the compilationMode to CompilationMode.Partial，other the same as origin project
AS: bumblebee

@Test
    fun startup() = benchmarkRule.measureRepeated(
        packageName = TARGET_PACKAGE,
        metrics = listOf(StartupTimingMetric()),
        compilationMode = CompilationMode.Partial(),
        iterations = 5,
        startupMode = StartupMode.COLD
    ) {
        pressHome()
        startActivityAndWait()
    }

Package x.y.z must not be running prior to cold start!

Hello,

I am trying to integrate Baseline Profiles but when I execute StartupBenchmark with COLD startup mode, it throws an exception

Package x.y.z must not be running prior to cold start!

Do you know why?

My config:

@RunWith(AndroidJUnit4::class)
class StartupBenchmark {

    @get:Rule
    val benchmarkRule = MacrobenchmarkRule()

    @Test
    fun startup() = benchmarkRule.measureRepeated(
        packageName = PACKAGE_NAME,
        metrics = listOf(StartupTimingMetric()),
        iterations = 5,
        startupMode = StartupMode.COLD
    ) {
        pressHome()
        startActivityAndWait()
    }
}

Document how to run on a single connceted device

Presumably with

ANDROID_SERIAL=XXXX

Otherwise tests run on all or a random device?

Should directory - BenchmarkSample be renamed as MicroBenchmarkSample ?

This is a nit:

Can this directory BenchmarkSample be renamed as MicroBenchmarkSample , that mirrors well with the new directory MacroBenchmarkSample?

java.lang.IllegalStateException: Unable to acquire intent for package com.xxx

java.lang.IllegalStateException: Unable to acquire intent for package com.xxx
at androidx.benchmark.macro.MacrobenchmarkScope.startActivityAndWait(MacrobenchmarkScope.kt:85)
at androidx.benchmark.macro.MacrobenchmarkScope.startActivityAndWait$default(MacrobenchmarkScope.kt:81)

What does "frameoverrunnanos" mean - JankStats

FrameDataApi31.java

Log : FrameData(frameStartNanos=769350421608955, frameDurationUiNanos=4858809, frameDurationCpuNanos=9390944, frameOverrunNanos=9222602686413166853, isJank=false, states=[])

isJank=false，but frameOverrunNanos is positive value

Macrobenchmark was crashed with following log

Tested version : 79a9e78
Tested devices :

Galaxy Fold Z 3 / Android 12 : samsung/q2qksx/qssi:12/SP1A.210812.016/F926NKSU1CVEE:user/release-keys
Xiaomi A1 / Android 9 : xiaomi/tissot/tissot_sprout:9/PKQ1.180917.001/V10.0.11.0.PDHMIXM:user/release-keys

Crash log:

 E AndroidRuntime: Process: com.example.macrobenchmark.target, PID: 10399
 E AndroidRuntime: java.lang.IllegalAccessError: Method 'int android.view.View.getWindowAttachCount()' is inaccessible to class 'android.view.ViewSpyInternalKt' (declaration of 'android.view.ViewSpyInternalKt' appears in /data/app/com.example.macrobenchmark.target-rAmBOWYiDAmM_yXDt20HxQ==/base.apk)
 E AndroidRuntime: 	at android.view.ViewSpyInternalKt.windowAttachCount(ViewSpyInternal.kt:7)
 E AndroidRuntime: 	at com.example.macrobenchmark.target.ClickTrace$install$1.onRootViewAdded(ClickTrace.kt:7)
 E AndroidRuntime: 	at com.example.macrobenchmark.target.ClickTrace$install$1.onRootViewsChanged(ClickTrace.kt:7)
 E AndroidRuntime: 	at curtains.internal.RootViewsSpy$delegatingViewList$1.add(RootViewsSpy.kt:28)
 E AndroidRuntime: 	at android.view.WindowManagerGlobal.addView(WindowManagerGlobal.java:350)
 E AndroidRuntime: 	at android.view.WindowManagerImpl.addView(WindowManagerImpl.java:94)
 E AndroidRuntime: 	at android.app.ActivityThread.handleResumeActivity(ActivityThread.java:3880)

Add missing copyright headers.

Some files that were added to the repository by Googlers are missing copyright headers. These need to be added.

FrameMetrics deviation

I'm observing strange results of ConstraintLayout measure + layout time.
According to FrameMetrics data of this benchmark, the ConstraintLayout's full measure + layout time for single iteration is 16040790ns (it varies, but let's take this as an example). Note, that this number is not for a single frame, but a sum of all frames data from the start to the end of the benchmark.

I was confused by these kind of results, because it's much less, than I would expect from layout of this complexity.

So I decided to add some manual logging of layout process, which looks like this:

val start = System.nanoTime()
measureAndLayoutExactLength(container)
measureAndLayoutWrapLength(container)
val end = System.nanoTime()
Log.d(TAG, "measure + layout: " + (end - start))

Which printed the result of 289000360ns for the exact same iteration. So the difference was 18 times!
Sometimes the deviation is less significant, like ~3 times, but FrameMetrics always give a much smaller number.

What's the reason for that and how this precision can be improved?

Exception during project sync

I'm trying to run macrobenchmark sample, but after cloning the repository and opening it in Android Studio I get an exception during sync:

FAILURE: Build failed with an exception.

* What went wrong:
com.intellij.openapi.externalSystem.model.ExternalSystemException (no error message)

Full output


FAILURE: Build failed with an exception.

* What went wrong:
com.intellij.openapi.externalSystem.model.ExternalSystemException (no error message)

* Try:
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Exception is:
com.intellij.openapi.externalSystem.model.ExternalSystemException: 
  at org.jetbrains.plugins.gradle.model.ProjectImportAction.addBuildModels(ProjectImportAction.java:346)
  at org.jetbrains.plugins.gradle.model.ProjectImportAction.execute(ProjectImportAction.java:127)
  at org.jetbrains.plugins.gradle.model.ProjectImportAction.execute(ProjectImportAction.java:42)
  at org.gradle.tooling.internal.consumer.connection.InternalBuildActionAdapter.execute(InternalBuildActionAdapter.java:64)
  at org.gradle.tooling.internal.provider.runner.AbstractClientProvidedBuildActionRunner$ActionAdapter.runAction(AbstractClientProvidedBuildActionRunner.java:131)
  at org.gradle.tooling.internal.provider.runner.AbstractClientProvidedBuildActionRunner$ActionAdapter.fromBuildModel(AbstractClientProvidedBuildActionRunner.java:104)
  at org.gradle.tooling.internal.provider.runner.AbstractClientProvidedBuildActionRunner$ActionAdapter.fromBuildModel(AbstractClientProvidedBuildActionRunner.java:84)
  at org.gradle.internal.buildtree.DefaultBuildTreeModelCreator.fromBuildModel(DefaultBuildTreeModelCreator.java:57)
  at org.gradle.internal.buildtree.DefaultBuildTreeLifecycleController.lambda$fromBuildModel$1(DefaultBuildTreeLifecycleController.java:82)
  at org.gradle.internal.buildtree.DefaultBuildTreeLifecycleController.lambda$runBuild$4(DefaultBuildTreeLifecycleController.java:106)
  at org.gradle.internal.model.StateTransitionController.lambda$transition$6(StateTransitionController.java:166)
  at org.gradle.internal.model.StateTransitionController.doTransition(StateTransitionController.java:238)
  at org.gradle.internal.model.StateTransitionController.lambda$transition$7(StateTransitionController.java:166)
  at org.gradle.internal.work.DefaultSynchronizer.withLock(DefaultSynchronizer.java:44)
  at org.gradle.internal.model.StateTransitionController.transition(StateTransitionController.java:166)
  at org.gradle.internal.buildtree.DefaultBuildTreeLifecycleController.runBuild(DefaultBuildTreeLifecycleController.java:103)
  at org.gradle.internal.buildtree.DefaultBuildTreeLifecycleController.fromBuildModel(DefaultBuildTreeLifecycleController.java:74)
  at org.gradle.tooling.internal.provider.runner.AbstractClientProvidedBuildActionRunner.runClientAction(AbstractClientProvidedBuildActionRunner.java:43)
  at org.gradle.tooling.internal.provider.runner.ClientProvidedPhasedActionRunner.run(ClientProvidedPhasedActionRunner.java:53)
  at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
  at org.gradle.internal.buildtree.ProblemReportingBuildActionRunner.run(ProblemReportingBuildActionRunner.java:49)
  at org.gradle.launcher.exec.BuildOutcomeReportingBuildActionRunner.run(BuildOutcomeReportingBuildActionRunner.java:69)
  at org.gradle.tooling.internal.provider.FileSystemWatchingBuildActionRunner.run(FileSystemWatchingBuildActionRunner.java:119)
  at org.gradle.launcher.exec.BuildCompletionNotifyingBuildActionRunner.run(BuildCompletionNotifyingBuildActionRunner.java:41)
  at org.gradle.launcher.exec.RootBuildLifecycleBuildActionExecutor.lambda$execute$0(RootBuildLifecycleBuildActionExecutor.java:40)
  at org.gradle.composite.internal.DefaultRootBuildState.run(DefaultRootBuildState.java:128)
  at org.gradle.launcher.exec.RootBuildLifecycleBuildActionExecutor.execute(RootBuildLifecycleBuildActionExecutor.java:40)
  at org.gradle.internal.buildtree.DefaultBuildTreeContext.execute(DefaultBuildTreeContext.java:40)
  at org.gradle.launcher.exec.BuildTreeLifecycleBuildActionExecutor.lambda$execute$0(BuildTreeLifecycleBuildActionExecutor.java:65)
  at org.gradle.internal.buildtree.BuildTreeState.run(BuildTreeState.java:53)
  at org.gradle.launcher.exec.BuildTreeLifecycleBuildActionExecutor.execute(BuildTreeLifecycleBuildActionExecutor.java:65)
  at org.gradle.launcher.exec.RunAsBuildOperationBuildActionExecutor$3.call(RunAsBuildOperationBuildActionExecutor.java:61)
  at org.gradle.launcher.exec.RunAsBuildOperationBuildActionExecutor$3.call(RunAsBuildOperationBuildActionExecutor.java:57)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
  at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
  at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
  at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
  at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
  at org.gradle.launcher.exec.RunAsBuildOperationBuildActionExecutor.execute(RunAsBuildOperationBuildActionExecutor.java:57)
  at org.gradle.launcher.exec.RunAsWorkerThreadBuildActionExecutor.lambda$execute$0(RunAsWorkerThreadBuildActionExecutor.java:36)
  at org.gradle.internal.work.DefaultWorkerLeaseService.withLocks(DefaultWorkerLeaseService.java:270)
  at org.gradle.internal.work.DefaultWorkerLeaseService.runAsWorkerThread(DefaultWorkerLeaseService.java:119)
  at org.gradle.launcher.exec.RunAsWorkerThreadBuildActionExecutor.execute(RunAsWorkerThreadBuildActionExecutor.java:36)
  at org.gradle.tooling.internal.provider.ContinuousBuildActionExecutor.execute(ContinuousBuildActionExecutor.java:103)
  at org.gradle.tooling.internal.provider.SubscribableBuildActionExecutor.execute(SubscribableBuildActionExecutor.java:64)
  at org.gradle.internal.session.DefaultBuildSessionContext.execute(DefaultBuildSessionContext.java:46)
  at org.gradle.tooling.internal.provider.BuildSessionLifecycleBuildActionExecuter$ActionImpl.apply(BuildSessionLifecycleBuildActionExecuter.java:100)
  at org.gradle.tooling.internal.provider.BuildSessionLifecycleBuildActionExecuter$ActionImpl.apply(BuildSessionLifecycleBuildActionExecuter.java:88)
  at org.gradle.internal.session.BuildSessionState.run(BuildSessionState.java:69)
  at org.gradle.tooling.internal.provider.BuildSessionLifecycleBuildActionExecuter.execute(BuildSessionLifecycleBuildActionExecuter.java:62)
  at org.gradle.tooling.internal.provider.BuildSessionLifecycleBuildActionExecuter.execute(BuildSessionLifecycleBuildActionExecuter.java:41)
  at org.gradle.tooling.internal.provider.StartParamsValidatingActionExecuter.execute(StartParamsValidatingActionExecuter.java:63)
  at org.gradle.tooling.internal.provider.StartParamsValidatingActionExecuter.execute(StartParamsValidatingActionExecuter.java:31)
  at org.gradle.tooling.internal.provider.SessionFailureReportingActionExecuter.execute(SessionFailureReportingActionExecuter.java:58)
  at org.gradle.tooling.internal.provider.SessionFailureReportingActionExecuter.execute(SessionFailureReportingActionExecuter.java:42)
  at org.gradle.tooling.internal.provider.SetupLoggingActionExecuter.execute(SetupLoggingActionExecuter.java:47)
  at org.gradle.tooling.internal.provider.SetupLoggingActionExecuter.execute(SetupLoggingActionExecuter.java:31)
  at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:65)
  at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:37)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:39)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:29)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.RequestStopIfSingleUsedDaemon.execute(RequestStopIfSingleUsedDaemon.java:35)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.create(ForwardClientInput.java:78)
  at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.create(ForwardClientInput.java:75)
  at org.gradle.util.internal.Swapper.swap(Swapper.java:38)
  at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:75)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.LogAndCheckHealth.execute(LogAndCheckHealth.java:55)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:63)
  at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:37)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:84)
  at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:37)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:52)
  at org.gradle.launcher.daemon.server.DaemonStateCoordinator$1.run(DaemonStateCoordinator.java:297)
  at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
java.lang.NullPointerException
  at com.android.tools.idea.gradle.project.sync.ModelCacheV1ImplKt.modelCacheV1Impl$sourceProviderFrom(ModelCacheV1Impl.kt:155)
  at com.android.tools.idea.gradle.project.sync.ModelCacheV1ImplKt.modelCacheV1Impl$buildTypeContainerFrom(ModelCacheV1Impl.kt:298)
  at com.android.tools.idea.gradle.project.sync.ModelCacheV1ImplKt.modelCacheV1Impl$androidProjectFrom$buildTypeContainerFrom-185(ModelCacheV1Impl.kt:1098)
  at com.android.tools.idea.gradle.project.sync.ModelCacheV1ImplKt.modelCacheV1Impl$androidProjectFrom(ModelCacheV1Impl.kt:1101)
  at com.android.tools.idea.gradle.project.sync.ModelCacheV1ImplKt.access$modelCacheV1Impl$androidProjectFrom(ModelCacheV1Impl.kt:1)
  at com.android.tools.idea.gradle.project.sync.ModelCacheV1ImplKt$modelCacheV1Impl$1.androidProjectFrom(ModelCacheV1Impl.kt:1179)
  at com.android.tools.idea.gradle.project.sync.AndroidExtraModelProviderWorkerKt.createAndroidModule(AndroidExtraModelProviderWorker.kt:786)
  at com.android.tools.idea.gradle.project.sync.AndroidExtraModelProviderWorkerKt.access$createAndroidModule(AndroidExtraModelProviderWorker.kt:1)
  at com.android.tools.idea.gradle.project.sync.AndroidExtraModelProviderWorker$populateAndroidModels$modules$1$1.invoke(AndroidExtraModelProviderWorker.kt:214)
  at com.android.tools.idea.gradle.project.sync.AndroidExtraModelProviderWorker$populateAndroidModels$modules$1$1.invoke(AndroidExtraModelProviderWorker.kt:153)
  at com.android.tools.idea.gradle.project.sync.SequentialSyncActionRunner.runAction(SyncActionRunner.kt:61)
  at com.android.tools.idea.gradle.project.sync.SequentialSyncActionRunner.runActions(SyncActionRunner.kt:57)
  at com.android.tools.idea.gradle.project.sync.AndroidExtraModelProviderWorker.populateAndroidModels(AndroidExtraModelProviderWorker.kt:151)
  at com.android.tools.idea.gradle.project.sync.AndroidExtraModelProviderWorker.populateBuildModels(AndroidExtraModelProviderWorker.kt:78)
  at com.android.tools.idea.gradle.project.sync.AndroidExtraModelProvider.populateBuildModels(AndroidExtraModelProvider.kt:59)
  at org.jetbrains.plugins.gradle.model.ProjectImportAction.addBuildModels(ProjectImportAction.java:334)
  at org.jetbrains.plugins.gradle.model.ProjectImportAction.execute(ProjectImportAction.java:127)
  at org.jetbrains.plugins.gradle.model.ProjectImportAction.execute(ProjectImportAction.java:42)
  at org.gradle.tooling.internal.consumer.connection.InternalBuildActionAdapter.execute(InternalBuildActionAdapter.java:64)
  at org.gradle.tooling.internal.provider.runner.AbstractClientProvidedBuildActionRunner$ActionAdapter.runAction(AbstractClientProvidedBuildActionRunner.java:131)
  at org.gradle.tooling.internal.provider.runner.AbstractClientProvidedBuildActionRunner$ActionAdapter.fromBuildModel(AbstractClientProvidedBuildActionRunner.java:104)
  at org.gradle.tooling.internal.provider.runner.AbstractClientProvidedBuildActionRunner$ActionAdapter.fromBuildModel(AbstractClientProvidedBuildActionRunner.java:84)
  at org.gradle.internal.buildtree.DefaultBuildTreeModelCreator.fromBuildModel(DefaultBuildTreeModelCreator.java:57)
  at org.gradle.internal.buildtree.DefaultBuildTreeLifecycleController.lambda$fromBuildModel$1(DefaultBuildTreeLifecycleController.java:82)
  at org.gradle.internal.buildtree.DefaultBuildTreeLifecycleController.lambda$runBuild$4(DefaultBuildTreeLifecycleController.java:106)
  at org.gradle.internal.model.StateTransitionController.lambda$transition$6(StateTransitionController.java:166)
  at org.gradle.internal.model.StateTransitionController.doTransition(StateTransitionController.java:238)
  at org.gradle.internal.model.StateTransitionController.lambda$transition$7(StateTransitionController.java:166)
  at org.gradle.internal.work.DefaultSynchronizer.withLock(DefaultSynchronizer.java:44)
  at org.gradle.internal.model.StateTransitionController.transition(StateTransitionController.java:166)
  at org.gradle.internal.buildtree.DefaultBuildTreeLifecycleController.runBuild(DefaultBuildTreeLifecycleController.java:103)
  at org.gradle.internal.buildtree.DefaultBuildTreeLifecycleController.fromBuildModel(DefaultBuildTreeLifecycleController.java:74)
  at org.gradle.tooling.internal.provider.runner.AbstractClientProvidedBuildActionRunner.runClientAction(AbstractClientProvidedBuildActionRunner.java:43)
  at org.gradle.tooling.internal.provider.runner.ClientProvidedPhasedActionRunner.run(ClientProvidedPhasedActionRunner.java:53)
  at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
  at org.gradle.internal.buildtree.ProblemReportingBuildActionRunner.run(ProblemReportingBuildActionRunner.java:49)
  at org.gradle.launcher.exec.BuildOutcomeReportingBuildActionRunner.run(BuildOutcomeReportingBuildActionRunner.java:69)
  at org.gradle.tooling.internal.provider.FileSystemWatchingBuildActionRunner.run(FileSystemWatchingBuildActionRunner.java:119)
  at org.gradle.launcher.exec.BuildCompletionNotifyingBuildActionRunner.run(BuildCompletionNotifyingBuildActionRunner.java:41)
  at org.gradle.launcher.exec.RootBuildLifecycleBuildActionExecutor.lambda$execute$0(RootBuildLifecycleBuildActionExecutor.java:40)
  at org.gradle.composite.internal.DefaultRootBuildState.run(DefaultRootBuildState.java:128)
  at org.gradle.launcher.exec.RootBuildLifecycleBuildActionExecutor.execute(RootBuildLifecycleBuildActionExecutor.java:40)
  at org.gradle.internal.buildtree.DefaultBuildTreeContext.execute(DefaultBuildTreeContext.java:40)
  at org.gradle.launcher.exec.BuildTreeLifecycleBuildActionExecutor.lambda$execute$0(BuildTreeLifecycleBuildActionExecutor.java:65)
  at org.gradle.internal.buildtree.BuildTreeState.run(BuildTreeState.java:53)
  at org.gradle.launcher.exec.BuildTreeLifecycleBuildActionExecutor.execute(BuildTreeLifecycleBuildActionExecutor.java:65)
  at org.gradle.launcher.exec.RunAsBuildOperationBuildActionExecutor$3.call(RunAsBuildOperationBuildActionExecutor.java:61)
  at org.gradle.launcher.exec.RunAsBuildOperationBuildActionExecutor$3.call(RunAsBuildOperationBuildActionExecutor.java:57)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
  at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
  at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
  at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
  at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
  at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
  at org.gradle.launcher.exec.RunAsBuildOperationBuildActionExecutor.execute(RunAsBuildOperationBuildActionExecutor.java:57)
  at org.gradle.launcher.exec.RunAsWorkerThreadBuildActionExecutor.lambda$execute$0(RunAsWorkerThreadBuildActionExecutor.java:36)
  at org.gradle.internal.work.DefaultWorkerLeaseService.withLocks(DefaultWorkerLeaseService.java:270)
  at org.gradle.internal.work.DefaultWorkerLeaseService.runAsWorkerThread(DefaultWorkerLeaseService.java:119)
  at org.gradle.launcher.exec.RunAsWorkerThreadBuildActionExecutor.execute(RunAsWorkerThreadBuildActionExecutor.java:36)
  at org.gradle.tooling.internal.provider.ContinuousBuildActionExecutor.execute(ContinuousBuildActionExecutor.java:103)
  at org.gradle.tooling.internal.provider.SubscribableBuildActionExecutor.execute(SubscribableBuildActionExecutor.java:64)
  at org.gradle.internal.session.DefaultBuildSessionContext.execute(DefaultBuildSessionContext.java:46)
  at org.gradle.tooling.internal.provider.BuildSessionLifecycleBuildActionExecuter$ActionImpl.apply(BuildSessionLifecycleBuildActionExecuter.java:100)
  at org.gradle.tooling.internal.provider.BuildSessionLifecycleBuildActionExecuter$ActionImpl.apply(BuildSessionLifecycleBuildActionExecuter.java:88)
  at org.gradle.internal.session.BuildSessionState.run(BuildSessionState.java:69)
  at org.gradle.tooling.internal.provider.BuildSessionLifecycleBuildActionExecuter.execute(BuildSessionLifecycleBuildActionExecuter.java:62)
  at org.gradle.tooling.internal.provider.BuildSessionLifecycleBuildActionExecuter.execute(BuildSessionLifecycleBuildActionExecuter.java:41)
  at org.gradle.tooling.internal.provider.StartParamsValidatingActionExecuter.execute(StartParamsValidatingActionExecuter.java:63)
  at org.gradle.tooling.internal.provider.StartParamsValidatingActionExecuter.execute(StartParamsValidatingActionExecuter.java:31)
  at org.gradle.tooling.internal.provider.SessionFailureReportingActionExecuter.execute(SessionFailureReportingActionExecuter.java:58)
  at org.gradle.tooling.internal.provider.SessionFailureReportingActionExecuter.execute(SessionFailureReportingActionExecuter.java:42)
  at org.gradle.tooling.internal.provider.SetupLoggingActionExecuter.execute(SetupLoggingActionExecuter.java:47)
  at org.gradle.tooling.internal.provider.SetupLoggingActionExecuter.execute(SetupLoggingActionExecuter.java:31)
  at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:65)
  at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:37)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:39)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:29)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.RequestStopIfSingleUsedDaemon.execute(RequestStopIfSingleUsedDaemon.java:35)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.create(ForwardClientInput.java:78)
  at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.create(ForwardClientInput.java:75)
  at org.gradle.util.internal.Swapper.swap(Swapper.java:38)
  at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:75)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.LogAndCheckHealth.execute(LogAndCheckHealth.java:55)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:63)
  at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:37)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:84)
  at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:37)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
  at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:52)
  at org.gradle.launcher.daemon.server.DaemonStateCoordinator$1.run(DaemonStateCoordinator.java:297)
  at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  at java.base/java.lang.Thread.run(Thread.java:829)



* Get more help at https://help.gradle.org

BUILD FAILED in 3s

But when I downgrade Android Gradle Plugin to version 7.2.0-alpha04 everything starts working.
Android Studio version: Android Studio Bumblebee | 2021.1.1 Patch 2

PS. I tried to create an issue on google issue tracker (link), but it says I don't have permission to create issues in AGP component

Timed out waiting for process (com.example.benchmark.test)

Specification
Android version: Android P
Android device: XIAOMI MI A1
Android Studio:

Android Studio 3.6 RC 1
Build #AI-192.7142.36.36.6071332, built on December 13, 2019
Runtime version: 1.8.0_212-release-1586-b4-5784211 x86_64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
macOS 10.14.6
GC: ParNew, ConcurrentMarkSweep
Memory: 8029M
Cores: 8

Steps to reproduce

clone this project
open the project
go to / create new benchmark test
run below script

benchmarkRule.measureRepeated {
	for (a in 0..10000) {
             Log.d("LogBenchmark", "the cost of writing this log method will be measured")
         }
}

Expected behavior

Testing started at 10.51 ...

12/31 10:51:37: Launching 'integerArtCacheAlloc()' on Xiaomi Mi A1.
Running tests

$ adb shell am instrument -w -r    -e debug false -e class 'com.example.benchmark.AutoBoxingBenchmark#integerArtCacheAlloc' com.example.benchmark.test/androidx.benchmark.junit4.AndroidBenchmarkRunner

Started running tests

benchmark:        X ns AutoBoxingBenchmark.integerArtCacheAlloc
Tests ran to completion.

X is a time

Actual behavior

Testing started at 10.52 ...

12/31 10:52:22: Launching 'integerArtCacheAlloc()' on Xiaomi Mi A1.
Running tests

$ adb shell am instrument -w -r    -e debug false -e class 'com.example.benchmark.AutoBoxingBenchmark#integerArtCacheAlloc' com.example.benchmark.test/androidx.benchmark.junit4.AndroidBenchmarkRunner

Started running tests

Timed out waiting for process (com.example.benchmark.test) to appear on xiaomi-mi_a1-00ccaa190904.

TrivialStartupBenchmark always failed for StartupMode.WARM and StartupMode.HOT

TrivialStartupBenchmark always failed for StartupMode.WARM and StartupMode.HOT. Also I noticed that all startup mode benchmark is working fine in androidx.benchmark:benchmark-macro-junit4:1.1.0-alpha02, but starting in androidx.benchmark:benchmark-macro-junit4:1.1.0-alpha05 the HOT and WARM startup mode always failed.

Error logs:
java.lang.IllegalArgumentException: Unable to read any metrics during benchmark (metric list: [androidx.benchmark.macro.StartupTimingMetric@b5f2a80]).
Check that you're performing the operations to be measured. For example, if
using StartupTimingMetric, are you starting an activity for the specified package
in the measure block?
at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmark(Macrobenchmark.kt:198)
at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmarkWithStartupMode(Macrobenchmark.kt:259)
at androidx.benchmark.macro.junit4.MacrobenchmarkRule.measureRepeated(MacrobenchmarkRule.kt:68)
at androidx.benchmark.macro.junit4.MacrobenchmarkRule.measureRepeated$default(MacrobenchmarkRule.kt:58)
at com.example.macrobenchmark.StartupUtilsKt.measureStartup(StartupUtils.kt:32)
at com.example.macrobenchmark.TrivialStartupBenchmark.startup(TrivialStartupBenchmark.kt:34)

Faling running macrobenchmark test samples

i'm encoutering an issues running the macrobenchmark on my pixel 3a.

i'm running : ./gradlew :macrobenchmark:connectedCheck

com.example.macrobenchmark.FrameTimingBenchmark / NonExportedActivityBenchmark

at androidx.benchmark.macro.JankCollectionHelper.getMetrics(JankCollectionHelper.java:227)
at androidx.benchmark.macro.FrameTimingMetric.getMetrics$benchmark_macro_release(Metric.kt:118)
at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmark(Macrobenchmark.kt:165)
at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmarkWithStartupMode(Macrobenchmark.kt:243)
at androidx.benchmark.macro.junit4.MacrobenchmarkRule.measureRepeated(MacrobenchmarkRule.kt:69)
at androidx.benchmark.macro.junit4.MacrobenchmarkRule.measureRepeated$default(MacrobenchmarkRule.kt:66)
at com.example.macrobenchmark.FrameTimingBenchmark.start(FrameTimingBenchmark.kt:45)
...

com.example.macrobenchmark.SmallListStartupBenchmark / TrivialStartupBenchmark

Check that you're performing the operations to be measured. For example, if
using StartupTimingMetric, are you starting an activity for the specified package
in the measure block?
at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmark(Macrobenchmark.kt:182)

com.example.macrobenchmark.TrivialStartupBenchmark
...

Unable to target the another package with in the project

java.lang.AssertionError: Unable to find target package org.view.activity, is it installed?
at androidx.benchmark.macro.MacrobenchmarkKt.checkErrors(Macrobenchmark.kt:48)
at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmark(Macrobenchmark.kt:126)
at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmarkWithStartupMode(Macrobenchmark.kt:301)
at androidx.benchmark.macro.junit4.MacrobenchmarkRule.measureRepeated(MacrobenchmarkRule.kt:102)
at androidx.benchmark.macro.junit4.MacrobenchmarkRule.measureRepeated$default(MacrobenchmarkRule.kt:92)
at com.ril.benchmark.VidyoActivityBenchmark.nonExportedActivityScrollList(ActivityBenchmark.kt:32)
... 34 trimmed
Caused by: android.content.pm.PackageManager$NameNotFoundException: org.jio.meet.conference.view.activity
at android.app.ApplicationPackageManager.getApplicationInfoAsUser(ApplicationPackageManager.java:452)
at android.app.ApplicationPackageManager.getApplicationInfo(ApplicationPackageManager.java:441)
at androidx.benchmark.macro.MacrobenchmarkKt.checkErrors(Macrobenchmark.kt:46)

ConstraintLayout vs. LinearLayout sample has nothing to do with Jetpack Benchmark

As a consumer of the Jetpack Benchmark docs, which led me to this repository, it's a little strange that that sample is included in this project. It does not appear to make use of the Jetpack Benchmark library and it took me a fair amount of fiddling to realize this (as that project does not build without upgrading various build tools.)

Perhaps it could be moved to another repository?

Can't setup macro-bench different product flavor (we cannot choose between the following variants of project )

How Can I setup macro-bench different product flavors (we cannot choose between the following variants of the project )

* What went wrong:
Could not determine the dependencies of task ':libraries:macrobenchmark:connectedBenchmarkAndroidTest'.
> Could not determine the dependencies of null.
   > Could not resolve all task dependencies for configuration ':libraries:macrobenchmark:benchmarkTestedApks'.
      > Could not resolve project :app.
        Required by:
            project :libraries:macrobenchmark
         > The consumer was configured to find a runtime of a component, as well as attribute

StartupMode.Cold will close app after 1 iteration

If you try scenario scrollNestedRecyclerWithoutRecyclerPool, it will just close app after 1 iteration.

I am using macrobenchmark version 1.2.0-alpha11.

This also happen to my app, is there bug with library ?

Jetpack Macrobenchmark takes a too long time to complete the test, its stuck in 97%

IDE info

Android Studio Dolphin | 2021.3.1
Build #AI-213.7172.25.2113.9014738, built on August 31, 2022
Runtime version: 11.0.13+0-b1751.21-8125866 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
Windows 10 10.0
GC: G1 Young Generation, G1 Old Generation
Memory: 2024M
Cores: 8
Registry:
    external.system.auto.import.disabled=true
    ide.text.editor.with.preview.show.floating.toolbar=false

Non-Bundled Plugins:
    wu.seal.tool.jsontokotlin (3.7.4)
    com.thoughtworks.gauge (213.5744.125)
    drawable.importer.pluginId.7658.hack' (0.8)
    de.ohmesoftware.parcelablegenerator (0.7.1)

I tried to test this sample app on my android device and emulator, it's taking a too long time (50 mins till now) to complete the test, it seems stuck on 97%

Starting a Gradle Daemon, 2 incompatible and 3 stopped Daemons could not be reused, use --status for details

Task :macrobenchmark:connectedBenchmarkAndroidTest
Starting 34 tests on MI 8 Lite - 11
<============-> 97% EXECUTING [50m 39s]
IDLE
IDLE
IDLE
:macrobenchmark:connectedBenchmarkAndroidTest

I've also created the issue here

Can macro benchmark run without clear app's data?

because in my app it require user to accept the license, and the macrobenchmark clear app's data every time when it startup the application.

Need examples on how to benchmark suspending functions in microbenchmark

Currently, when benchmarking a suspending function, I do the following:

    val benchmarkRule = BenchmarkRule()

    @Test
    fun benchmarkSubject() = benchmarkRule.measureRepeated {
        runBlocking {
            subject.suspendingFunction()
        }
    }

Is this the best way to do it? If so, can there be a helper extension like benchmarkRule.measureSuspendingRepeated?

Macrobenchmark is failing due to null result in clickOnId

One of classes includes following method :

private fun MacrobenchmarkScope.clickOnId(resourceId: String) {
        val selector = By.res(packageName, resourceId)
        if (!device.wait(Until.hasObject(selector), 2_500)) {
            fail("Did not find object with id $resourceId")
        }

        device
            .findObject(selector)
            .click()
        // Chill to ensure we capture the end of the click span in the trace.
        Thread.sleep(100)
    }

And it has been called like : clickOnId("launchRecyclerActivity")

The problem is this line returns null : val selector = By.res(packageName, resourceId)

And therefore I receive following exception :

java.lang.NullPointerException: Attempt to invoke virtual method 'void androidx.test.uiautomator.UiObject2.click()' on a null object reference
at com.example.macrobenchmark.clickslatency.ClickLatencyBenchmark.clickOnId(ClickLatencyBenchmark.kt:189)
at com.example.macrobenchmark.clickslatency.ClickLatencyBenchmark.access$clickOnId(ClickLatencyBenchmark.kt:40)
at com.example.macrobenchmark.clickslatency.ClickLatencyBenchmark$simpleViewClick$2.invoke(ClickLatencyBenchmark.kt:64)
at com.example.macrobenchmark.clickslatency.ClickLatencyBenchmark$simpleViewClick$2.invoke(ClickLatencyBenchmark.kt:51)
at androidx.benchmark.macro.MacrobenchmarkKt$macrobenchmark$measurements$1$tracePath$1.invoke(Macrobenchmark.kt:193)
at androidx.benchmark.macro.MacrobenchmarkKt$macrobenchmark$measurements$1$tracePath$1.invoke(Macrobenchmark.kt:167)
at androidx.benchmark.perfetto.PerfettoCaptureWrapper.record(PerfettoCaptureWrapper.kt:90)
at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmark(Macrobenchmark.kt:167)
at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmarkWithStartupMode(Macrobenchmark.kt:301)
at androidx.benchmark.macro.junit4.MacrobenchmarkRule.measureRepeated(MacrobenchmarkRule.kt:106)
at com.example.macrobenchmark.clickslatency.ClickLatencyBenchmark.simpleViewClick(ClickLatencyBenchmark.kt:51)

I also have this issue in NestedRecyclerFrameTimingBenchmarks class as well. How it can be resolved?

If I replace it with following method, it works as expected :

private fun MacrobenchmarkScope.clickOnText(text: String) {
device
.findObject(By.text(text))
.click()
}

Macroprofile doesn't work with Android Studio BumbleBee

The documentation says one can use Android Studio BumbleBee to build project, but MacroProfile doesn't work because of android gradle plugin version 7.2.0-beta04, if one downgrades to plugin 7.0.X project at least builds.

Does one need to use 7.2.0-beta04 for Macroprofile to work properly?

Profile is empty exception shown while generating baseline profile

I ran the generate class BaselineProfileGenerator in Android 11, non Google play service rooted device.

This the exception I get.

09-18 10:28:55.871 9313 9335 E TestRunner: java.lang.IllegalStateException: The profile is empty.
09-18 10:28:55.871 9313 9335 E TestRunner: at androidx.benchmark.macro.BaselineProfilesKt.profile(BaselineProfiles.kt:117)
09-18 10:28:55.871 9313 9335 E TestRunner: at androidx.benchmark.macro.BaselineProfilesKt.collectBaselineProfile(BaselineProfiles.kt:81)
09-18 10:28:55.871 9313 9335 E TestRunner: at androidx.benchmark.macro.junit4.BaselineProfileRule.collectBaselineProfile(BaselineProfileRule.kt:89)
09-18 10:28:55.871 9313 9335 E TestRunner: at com.example.macrobenchmark.BaselineProfileGenerator.generate(BaselineProfileGenerator.kt:40)
09-18 10:28:55.871 9313 9335 E TestRunner: at java.lang.reflect.Method.invoke(Native Method)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
09-18 10:28:55.871 9313 9335 E TestRunner: at androidx.benchmark.macro.junit4.BaselineProfileRule$applyInternal$1.evaluate(BaselineProfileRule.kt:76)
09-18 10:28:55.871 9313 9335 E TestRunner: at androidx.test.rule.GrantPermissionRule$RequestPermissionStatement.evaluate(GrantPermissionRule.java:134)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.rules.RunRules.evaluate(RunRules.java:20)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
09-18 10:28:55.871 9313 9335 E TestRunner: at androidx.test.ext.junit.runners.AndroidJUnit4.run(AndroidJUnit4.java:162)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.Suite.runChild(Suite.java:128)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.Suite.runChild(Suite.java:27)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
09-18 10:28:55.871 9313 9335 E TestRunner: at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
09-18 10:28:55.871 9313 9335 E TestRunner: at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
09-18 10:28:55.871 9313 9335 E TestRunner: at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:444)
09-18 10:28:55.871 9313 9335 E TestRunner: at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2205)
09-18 10:28:55.871 9313 9335 E TestRunner: ----- end exception -----
09-18 10:28:55.872 9313 9335 I TestRunner: finished: generate(com.example.macrobenchmark.BaselineProfileGenerator)

java.lang.IllegalStateException: The profile is empty.
at androidx.benchmark.macro.BaselineProfilesKt.profile(BaselineProfiles.kt:117)
at androidx.benchmark.macro.BaselineProfilesKt.collectBaselineProfile(BaselineProfiles.kt:81)
at androidx.benchmark.macro.junit4.BaselineProfileRule.collectBaselineProfile(BaselineProfileRule.kt:89)
at com.example.macrobenchmark.BaselineProfileGenerator.generate(BaselineProfileGenerator.kt:40)

The Sample Project uses MacroBench Version 1.1.0-beta06 and minimum sdk version 21

The Sample Project uses MacroBench Version 1.1.0-beta06 and minimum sdk version 21, while using the same MacroBench Version and minimum sdk version I'm getting the error:

 Manifest merger failed : uses-sdk:minSdkVersion 21 cannot be smaller than version 23 declared in library [androidx.benchmark:benchmark-macro-junit4:1.1.0-beta06] /Users/raghav/.gradle/caches/transforms-3/78a0d0ee055021932fb0b29510d878d4/transformed/jetified-benchmark-macro-junit4-1.1.0-beta06/AndroidManifest.xml as the library might be using APIs not available in 21
	Suggestion: use a compatible library with a minSdk of at most 21,
		or increase this project's minSdk version to at least 23,
		or use tools:overrideLibrary="androidx.benchmark.macro.junit4" to force usage (may lead to runtime failures)

Need Example using Dagger2 or Hilt

There needs to be an example of using the Microbench Marking with Dagger or Hilt dependencies. Current examples don't really go into modern development practices.

Unable to successfully run macrobenchmark test based off on sample

I'm trying to run a simple startup test similar to the one shown in the sample here, and although I seem to have everything as shown in the sample gradle file, I keep getting an error when running the test. I'm trying to run directly on a FB device through the gcloud command, using the same command shown in the sample, and although the test seems to start correctly, I keep seeing the following:

java.lang.NoClassDefFoundError: Failed resolution of: Lkotlin/collections/CollectionsKt; at com.sample.macrotests.ColdStartupTest.startup(ColdStartupTest.kt:23) ... 30 trimmed Caused by: java.lang.ClassNotFoundException: kotlin.collections.CollectionsKt ... 32 more

this points to the line where I'm just setting the type of test to run, like:

metrics = listOf(StartupTimingMetric()),

I seem to be missing a dependency but I have everything as shown in the sample so I haven't been able to figure out exactly what I'm missing so not sure if it's a real issue or just something on my side I may be missing so just wanted to call it out here.

Thanks!

Could not validate the dex paths: Unmatched .dm files: [release.dm]

I follow https://developer.android.google.cn/topic/performance/baselineprofiles#measuring-improvements to measure the improvement, but got warning when doing :
% adb install-multiple app-release.apk release.dm
adb: failed to finalize session
Warning [Could not validate the dex paths: Unmatched .dm files: [release.dm]]
Success

Does it mean fail to sideload the Baseline Profile ? how can i fix this ? please help ~

My adb version:
Android Debug Bridge version 1.0.41
Version 31.0.2-7242960
Installed as /usr/local/bin/adb

Unable to read any metrics during benchmark

androidx.benchmark:benchmark-macro-junit4:1.1.0-beta04

java.lang.IllegalArgumentException: Unable to read any metrics during benchmark (metric list: [androidx.benchmark.macro.StartupTimingMetric@ac86f88]).
Check that you're performing the operations to be measured. For example, if
using StartupTimingMetric, are you starting an activity for the specified package
in the measure block?
	at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmark(Macrobenchmark.kt:237)
	at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmarkWithStartupMode(Macrobenchmark.kt:301)
	at androidx.benchmark.macro.junit4.MacrobenchmarkRule.measureRepeated(MacrobenchmarkRule.kt:102)
	at androidx.benchmark.macro.junit4.MacrobenchmarkRule.measureRepeated$default(MacrobenchmarkRule.kt:92)
	at com.example.benchmark.BaselineProfileBenchmark.startup(BaselineProfileBenchmark.kt:40)
	at com.example.benchmark.BaselineProfileBenchmark.startupNoCompilation(BaselineProfileBenchmark.kt:31)

Re-enable ScrollBenchmarks

Currently disabled as we can't see whether there is a scrollable view on screen.
If there's none this causes many failures, slowing down the process.
To re-enable we need 1) a way to check if a scrollable view is present 2) run the benchmarks only if 1) is true.

Add a composition tracing sample

Add a composition tracing sample, particularly with macrobenchmark.

https://developer.android.com/jetpack/compose/performance/tracing

Broadcast not received

I am currently using profileinstaller 1.3.0-alpha05 and benchmark 1.2.0-alpha08 in order to trigger baseline generator on A13 device without root. After upgrading the benchmark, I am not able to execute any macrobenchmark test and getting the below error

java.lang.IllegalStateException: The DROP_SHADER_CACHE broadcast was not received. This most likely means that the androidx.profileinstaller library used by the target apk is old. Please use 1.3.0-alpha02 or newer.

If i downgrade the benchmark to 1.1.1 and trigger the same macrobenchmark test then it's working fine.

Cannot run example in Android Studio 3.4.2

When I click the green triangle button to run "All in benchmark" in GUI, I'm getting

"/Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java" -ea -...
!!! JUnit version 3.8 or later expected:

java.lang.ClassNotFoundException: junit.framework.ComparisonFailure
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at com.intellij.rt.execution.junit.JUnitStarter.junitVersionChecks(JUnitStarter.java:221)
	at com.intellij.rt.execution.junit.JUnitStarter.canWorkWithJUnitVersion(JUnitStarter.java:207)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:61)

Process finished with exit code 253

I tried to Run -> Edit Configurations' Android JUnit to JRE 1.8 (on my Mac) and also to API Level 28.

Cloud function unable to find the generated .json file

I have a little confusion:
What should be the environment variable which specifies target package? In the sample workflow provided under ftl package, the target package environment variable is set to com.example.macrobenchmark which is the package name of the macrobenchmark module which contains test classes not the app module which contains classes that have to be benchmarked.

One has to generate 2 apks as per documentation, one which is as close to release build as possible (init with release and signingconfig = debug) and other apk is the benchmark apk. I'm doing this in my CI by :app:assemble[VariantName] and :macrobenchmark:assemble[VariantName] respectively.

say my app has package com.raghav
and the benchmark apk has the package name com.raghav.benchmark

Which of these has to be used in the package name environment variable?
Also while configuring CI action for running tests what apk path has to be provided for --app and --test
and it is mentioned to provide the path to external storage to store and retrieve the json file, if i'm not providing it the json file is not getting generated at all if am I providing it my cloud function is not looking for that file inside the provided path and giving the error 'Unable to locate the object'.

The main reason I find for the whole confusion is that the sample images provided in the readme under ftl package are not matching with the contents of that file.
Thanks in advance :)

How to findObject if my applicationId and packageName name of application is different

Eg:
Application_Id=com.raju.center (this is the application I am trying to benchmark) but my package name is com.app.center inside that package, I have an activity that has a fragment that contains the Recycler view I want to access to recycleViewId but it gives null pointer exception because this code val recycler = device.findObject(By.res(packageName, "recycler")) looking inside com.raju.center but my code package structure is different I also tried using a direct class-level package name it doesn't work either also tried adding content description it doesn't work for some reason. I don't want to use val recycler = device.findObject(By.scrollable(true)) for now

How to find viewObject ? (In the performance sample all the activity is in the same package and have the same application id and package name ) I tried changing the sample with different applicationId and package names those tests are failing

       val recycler = device.findObject(By.res(packageName, "recycler"))
        // Set gesture margin to avoid triggering gesture navigation
        // with input events from automation.
        recycler.setGestureMargin(device.displayWidth / 5) //getting null pointer exception

        // Scroll down several times
        repeat(3) { recycler.fling(Direction.DOWN) }

please retry with version 2020.3.1 or newer.

This version of the Android Support plugin for IntelliJ IDEA (or Android Studio) cannot open this project, please retry with version 2020.3.1 or newer.