takahirom / roborazzi Goto Github PK

This is reported in manifest merging

> Task :samples:star:processReleaseUnitTestManifest
[io.github.takahirom.roborazzi:roborazzi-junit-rule:1.1.0-alpha-3] /Users/zacsweers/.gradle/caches/transforms-3/7285b90af791688e44c5e5722003c07e/transformed/roborazzi-junit-rule-1.1.0-alpha-3/AndroidManifest.xml Warning:
        Namespace 'com.github.takahirom.roborazzi' used in: io.github.takahirom.roborazzi:roborazzi-junit-rule:1.1.0-alpha-3, io.github.takahirom.roborazzi:roborazzi:1.1.0-alpha-3.

A project with compose enabled can have Screens written using view-system as well. Why do we need to use createAndroidComposeRule<Activity>() to capture screen level image even when the activity's UI is written using view system?

In a compose enabled project, calling

@Test
  @Config(qualifiers = "+land")
  fun captureRoboImageSample() {
    // launch
    ActivityScenario.launch(MainActivity::class.java)
    // screen level image
    onView(ViewMatchers.isRoot())
      .captureRoboImage()
  }

ends up with following crash

java.lang.NullPointerException
	at com.github.takahirom.roborazzi.RoboCanvas.save(RoboCanvas.kt:195)
	at com.github.takahirom.roborazzi.RoborazziKt.processOutputImageAndReport(Roborazzi.kt:653)
	at com.github.takahirom.roborazzi.RoborazziKt.access$processOutputImageAndReport(Roborazzi.kt:1)
	at com.github.takahirom.roborazzi.RoborazziKt$captureRoboImage$1.invoke(Roborazzi.kt:99)
	at com.github.takahirom.roborazzi.RoborazziKt$captureRoboImage$1.invoke(Roborazzi.kt:98)
	at com.github.takahirom.roborazzi.RoborazziKt.capture(Roborazzi.kt:707)
	at com.github.takahirom.roborazzi.ImageCaptureViewAction.perform(Roborazzi.kt:681)
	at androidx.test.espresso.ViewInteraction$SingleExecutionViewAction.perform(ViewInteraction.java:2)
	at androidx.test.espresso.ViewInteraction.doPerform(ViewInteraction.java:25)
	at androidx.test.espresso.ViewInteraction.-$$Nest$mdoPerform(ViewInteraction.java)
	at androidx.test.espresso.ViewInteraction$1.call(ViewInteraction.java:7)
	at androidx.test.espresso.ViewInteraction$1.call(ViewInteraction.java:1)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at android.os.Handler.$$robo$$android_os_Handler$handleCallback(Handler.java:938)
	at android.os.Handler.handleCallback(Handler.java)
	at android.os.Handler.$$robo$$android_os_Handler$dispatchMessage(Handler.java:99)
	at android.os.Handler.dispatchMessage(Handler.java)
	at org.robolectric.shadows.ShadowPausedLooper$IdlingRunnable.run(ShadowPausedLooper.java:368)
	at org.robolectric.shadows.ShadowPausedLooper.executeOnLooper(ShadowPausedLooper.java:402)
	at org.robolectric.shadows.ShadowPausedLooper.idle(ShadowPausedLooper.java:93)
	at org.robolectric.android.internal.LocalControlledLooper.drainMainThreadUntilIdle(LocalControlledLooper.java:18)
	at androidx.test.espresso.ViewInteraction.waitForAndHandleInteractionResults(ViewInteraction.java:1)
	at androidx.test.espresso.ViewInteraction.desugaredPerform(ViewInteraction.java:11)
	at androidx.test.espresso.ViewInteraction.perform(ViewInteraction.java:8)
	at com.github.takahirom.roborazzi.RoborazziKt.captureRoboImage(Roborazzi.kt:98)
	at com.github.takahirom.roborazzi.RoborazziKt.captureRoboImage(Roborazzi.kt:87)
	at com.github.takahirom.roborazzi.RoborazziKt.captureRoboImage$default(Roborazzi.kt:82)

Is there a way to specify separate directories for the golden screenshots and the comparison images?

Background

In our project, we use the outputDirectoryPath option to specify where golden screenshots should be stored. Currently, we specify a directory outside of /build and commit the generated files to Git using Git LFS. Here's an example:

class RuleTestWithPath {
  @get:Rule
  val roborazziRule = RoborazziRule(
    options = Options(
      outputDirectoryPath = "screenshots/custom_outputDirectoryPath",
      ...
    ),
  )

However, one limitation of using outputDirectoryPath is that the images generated by verifyRoborazziDebug for failing tests will also be put into the same folder:

While keeping the golden screenshots stored in the path specified by outputDirectoryPath, we would prefer to have the comparison images located inside the /build folder. This approach offers the following advantages:

• It helps to prevent accidental commits of the comparison files to Git, eliminating the need for manual deletion of the comparison files.
• When golden screenshots and comparison files are already separated, it becomes easier to store only the comparison images in our CI system for reviewing each run.

This would be an essential feature, as users would not be able to change the size without this feature when using the velify task.
I am considering a parameter that allows the user to pass a value of 0~1, so that passing 0.5 would result in 50%.

resize = 0.5

Hi there,

I have been playing around this library by checking out the whole repo locally and running it locally. I also made effort in creating a full isolated app to play around the library.

However, I could not figure out the effectiveness of this library.
To start with, I could not find any snapshot view generated under build/outputs/roborazzi .
Then , how would it catch a bug if I purposefully break the code? Running ./gradlew recordRoborazziDebug or ./gradlew verifyRoborazziDebug does not break at all matter how I change the code.

I think there must be gap that I failed to interpret it by following the readme.

I tried to find any other sample app that is using Roborazzi. But I only find coil library, which is large. Hence, I did not explore it further.
Could you kindly check out this sample here https://github.com/TonyTangAndroid/HelloRoborazzi and see to the gap?

It would be great that we could have a minimum app to demonstrate how Roborazzi is working.

The file could be saved in ~/.gradle/daemon/7.6/build/outputs/roborazzi/xxx.png or project/build/outputs/roborazzi/xxx.png.
It should be a project path.

I don't know if I do this or not, but We should be able to test both the JVM and the emulator/device using the same test with the following steps using AndroidJUnit4.

• Create Gradle task for AndroidTest
• Sending and receiving images via adb. Before the test starts, push the image that is currently stored to the device for verify. And pull the results when the test is over.
• Creation of roborazzi-android-painter and roborazzi-awt-painter. We need to abstract the painter. We need to be able to draw to Android Bitmap what we are doing now with BufferedImage.

I'm trying to replace our exiting Paparazzi snapshot testing with Roborazzi.
One of important requirements for us is to test different font scales (1.0 to 2.0; up to 200% to meet WCAG guideline).
Paparazzi provides a way to do like below for example, but I don't seem to find an available RoborazziOptions.
Could you please tell me if there is an option to try out? If not, can we please add it to test a11y? Thanks.
@get:Rule val paparazzi = Paparazzi( deviceConfig = DeviceConfig.PIXEL_5.copy(fontScale = 2.0f))

It would be nice to be able to set some alternative strategy instead of DefaultFileNameGenerator

Particularly when transitioning from another test library.

The issue

Currently, Roborazzi depends heavily on Espresso and ComposeRule.

That's not the case with other screenshot testing frameworks, like Paparazzi, Shot, Dropshots...

In screenshot testing, it's also common to inflate a View without attaching it to an Activity. If not attached, you cannot find it with Espresso, and therefore it would not be possible to screenshot it.

Moreover, not attaching a View to an Activity has one extra advantage:
you can control its width and height to see how the View would render for different widths and heights.

That's something AndroidUiTestingUtils provides via waitForMeasureView(exactWidthPx, exactHeightPx), like in this example here, but it's not compatible with Roborazzi because of the reason I've described above.

Desired Solution

Ideally, captureRoboImage(view) and captureRoboImage(@Composable () -> Unit) should be provided to enable such cases and become less Espresso/ComposeRule dependent.

Additionally, a similar option for bitmaps e.g. captureRoboImage(bitmap) would be also useful, since you can draw a bitmap out of a View/Composable but applying your own Bitmap options. Shot and Dropshots offer that as well.

Setting ImageView.setDrawable isn't reflected in the screenshot. Here's a test that reproduces the issue. The test updates the image view's drawable to a red box, but the resulting screenshot is empty. Interestingly, using imageView.drawToBitmap renders the red square correctly.

Also is it possible to change the size of the screenshot? Thanks!

I have a fairly simple test that presses some buttons to toggle between weeks and update some text to reflect the new week.
When I add Roborazzi, the test fails with the error message Iteration already started. I've also noticed that the test times are a lot longer than without Roborazzi (see attached images).

@RunWith(RobolectricTestRunner::class)
@GraphicsMode(GraphicsMode.Mode.NATIVE)
class CalendarUiTests {
    @get:Rule
    val composeTestRule = createAndroidComposeRule<ComponentActivity>()

    @get:Rule
    val roborazziRule = RoborazziRule(
        composeRule = composeTestRule,
        captureRoot = composeTestRule.onRoot(),
        options = RoborazziRule.Options(
            captureType = RoborazziRule.CaptureType.LastImage,
        )
    )

    @Before
    @Throws(Exception::class)
    fun setup() {
        ShadowLog.stream = System.out
    }

    @Test
    fun `calendar top bar changes dates`() {
        var start by mutableStateOf(LocalDateTime.of(
            LocalDate.of(2023, 10, 1),
            LocalTime.MIDNIGHT
        ))
        var end by mutableStateOf(LocalDateTime.of(
            LocalDate.of(2023, 10, 7),
            LocalTime.MAX
        ))

        composeTestRule.setContent {
            CalendarTopBar(
                start = start,
                end = end,
                onPreviousWeek = {
                    start = start.minus(7, ChronoUnit.DAYS)
                    end = end.minus(7, ChronoUnit.DAYS)
                },
                onNextWeek = {
                    start = start.plus(7, ChronoUnit.DAYS)
                    end = end.plus(7, ChronoUnit.DAYS)
                }
            )
        }

        composeTestRule.onRoot().printToLog("TAG")

        composeTestRule.onNodeWithTag("calendarTitleBar")
            .assertIsDisplayed()
            .assertTextContains("Oct 1 - Oct 7")

        composeTestRule.onNodeWithTag("buttonPreviousWeek")
            .performClick()

        composeTestRule.onRoot().printToLog("TAG")

        composeTestRule.onNodeWithTag("calendarTitleBar")
            .assertIsDisplayed()
            .assertTextContains("Sep 24 - Sep 30")

        composeTestRule.onNodeWithTag("buttonNextWeek")
            .performClick()
            .performClick() // Fails on this line

        composeTestRule.onNodeWithTag("calendarTitleBar")
            .assertIsDisplayed()
            .assertTextContains("Oct 8 - Oct 14")
    }

}

If I remove Roborazzi, the test passes with no issues.

CalendarTopBar for reference:

@Composable
fun CalendarTopBar(
    start: LocalDateTime,
    end: LocalDateTime,
    onPreviousWeek: () -> Unit,
    onNextWeek: () -> Unit
) {
    val dtf = DateTimeFormatter.ofPattern("MMM d")
    Row(
        modifier = Modifier.padding(10.dp),
        verticalAlignment = Alignment.CenterVertically
    ) {
        Button(
            modifier = Modifier.testTag("buttonPreviousWeek"),
            onClick = {
                onPreviousWeek()
            },
            shape = RoundedCornerShape(2.dp)
        ) {
            Image(
                colorFilter = ColorFilter.tint(MaterialTheme.colorScheme.onPrimary),
                contentDescription = null,
                painter = painterResource(id = R.drawable.ic_arrow_back),
            )
        }
        Spacer(modifier = Modifier.weight(1f))
        Text(
            modifier = Modifier.testTag("calendarTitleBar"),
            style = MaterialTheme.typography.titleLarge,
            text = "${start.format(dtf)} - ${end.format(dtf)}"
        )
        Spacer(modifier = Modifier.weight(1f))
        Button(
            modifier = Modifier.testTag("buttonNextWeek"),
            onClick = {
                onNextWeek()
            },
            shape = RoundedCornerShape(2.dp)
        ) {
            Image(
                colorFilter = ColorFilter.tint(MaterialTheme.colorScheme.onPrimary),
                contentDescription = null,
                painter = painterResource(id = R.drawable.ic_arrow_forward),
            )
        }
    }
}

Test times without Roborazzi:

Test times with Roborazzi:

For example, when verifying it seems like there are two things that end up in the build/outputs/roborazzi directory:

• The original image that was recorded in a previous "record" step
• A comparison image that compares the recorded image above with the new/actual image

It would be nice if the "new/actual" image could also be provided separately. This way we can easily use it as a new baseline recorded image if we wish (i.e. if the changes in the test are actually intentional and we want to update our "golden value" image to use the next time the test is run).

Most screenshot testing frameworks provide some “tolerance” factor: the test will still pass even though a given percentage of pixels differ.

Note that this might also happen on JVM tests, for instance, when running them on different Operating Systems
cashapp/paparazzi#311

In fact, Paparazzi’s default tolerance is 0.1 (10% might differ and it would still pass)

Currently users of the library should manually add necessary dependencies, such as roborazzi artifacts and robolectric.
My suggestion is to make RoborazziPlugin responsible for managing dependencies.
I see roborazzi is split into 3 artifacts, so plugin could install at least core artifact and robolectric.
I can work on this, if you ok with external contributions

4 problems were found storing the configuration cache, 3 of which seem unique.

• Task :app:compareRoborazziDebug of type io.github.takahirom.roborazzi.RoborazziPlugin$CompareReportGenerateTask: invocation of 'Task.project' at execution time is unsupported.
  See https://docs.gradle.org/7.6.1/userguide/configuration_cache.html#config_cache:requirements:use_project_during_execution

Maybe it occurred here
https://github.com/takahirom/roborazzi/blob/main/roborazzi-gradle-plugin/src/main/java/io/github/takahirom/roborazzi/RoborazziPlugin.kt#L124

I initially thought it would be good to use "_" instead of "." in the file names. However, I'm now beginning to think it might be better to leave the names as they are because the JUnit report's file name will remain unchanged. This could be a breaking change, and we may need to provide some compatibility options.

If you are okay with a quick fix even if it's a breaking change, please react with 🚀
If you would like it to be an option but don't mind a slower fix, please react with 🎉
If you think no changes are needed, please react with 👀

It is a pain to name a file every time I call a function, so I want to make it so that I don't have to name it somehow.
If possible, I would like to get the name of the method of the test well.

Low priority feature request. Something like

https://github.com/google/android-wear-stitch-script

Currently, one can run Roborazzi tests only by running a couple of custom tasks, which are mainly a copy paste from Paparazzi.

I believe this is not necessary and would be awesome to be able to additionally run tests directly from Android Studio.

The idea is to do sth similar to Dropshots:

• the verification of the screenshots happens inside the test itself, and not after all screenshots are recorded. Not sure if that’s how it is currently done.
• Instead of having a custom task to run tests (:recordRoborazziTest/:verifyRoborazziTest) one would reuse the already existing task for that (:testDebugUnitTest), and use a gradle property to decide whether to record or verify the test e.g roborazzi.record

Dropshots does that with :connectedAndroidTest, that is the analogue task for instrumentation tests, and that allows to run the screenshot tests directly from Android studio

It would be great if the following requirements are also noted in README to introduce this plugin to several projects.

• Android Gradle Plugin version
• Gradle version
• JRE version

I found the min version of Robolectric (it's 4.10 alpha or later) by the way.

This repository is obviously a fork of Paparazzi, but contains none of the upstream license headers.

Now, we have support for difference-based GIF images that record when the image changes, making it unsuitable for checking animations.

#52 (comment)

#55

Additional action of task ':lazycolumnscreen:crosslibrary:testDebugUnitTest' was implemented by the Java lambda 'io.github.takahirom.roborazzi.RoborazziPlugin$apply$1$testTaskProvider$1$Lambda$2276/0x000000080268ba78'. Reason: Using Java lambdas is not supported as task inputs. Please refer to https://docs.gradle.org/7.5/userguide/validation_problems.html#implementation_unknown for more details about this problem.

Hi, I'm getting the following error when updating from 1.0.0-rc-3 to 1.1.0:

java.lang.AssertionError: Roborazzi: /Users/colinwhite/coil/coil-test-roborazzi/src/test/snapshots/images/coil_test_RoborazziViewTest_loadView_compare.png is added.
See compare image at /Users/colinwhite/coil/coil-test-roborazzi/src/test/snapshots/images/coil_test_RoborazziViewTest_loadView_compare.png
	at com.github.takahirom.roborazzi.RoborazziOptions$RoborazziCompareReporter$VerifyRoborazziCompareReporter.report(capture.kt:270)
	at com.github.takahirom.roborazzi.RoborazziKt.saveOrCompare(Roborazzi.kt:601)
	at com.github.takahirom.roborazzi.RoborazziKt.saveLastImage(Roborazzi.kt:441)
	at com.github.takahirom.roborazzi.RoborazziKt.access$saveLastImage(Roborazzi.kt:1)
	at com.github.takahirom.roborazzi.RoborazziKt$captureAndroidView$3.invoke(Roborazzi.kt:419)
	at com.github.takahirom.roborazzi.RoborazziKt$captureAndroidView$3.invoke(Roborazzi.kt:418)
	at com.github.takahirom.roborazzi.RoborazziRule$apply$1.evaluate(RoborazziRule.kt:125)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.robolectric.RobolectricTestRunner$HelperTestRunner$1.evaluate(RobolectricTestRunner.java:589)
	at org.robolectric.internal.SandboxTestRunner$2.lambda$evaluate$2(SandboxTestRunner.java:290)
	at org.robolectric.internal.bytecode.Sandbox.lambda$runOnMainThread$0(Sandbox.java:99)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

If I generate the _compare screenshots with the new compare task, the verify tasks still fails with the same error. Here's a PR that reproduces the issue by running: ./gradlew verifyRoborazziDebug --rerun-tasks

Repro: slackhq/circuit#669

Run the PetListSnapshotTest.petList_show_list_for_success_state() test, observe that it always fails locally and cannot record a new snapshot. On CI however, the verify calls always pass despite there being no available snapshot.

This is using 1.3.0-alpha-4

The Rule has that process, but the Manual Test does not.

Useful to show configuring the screen for different devices. Round vs Square watch.

These qualifiers should also affect which resources are loaded

@Config(
    sdk = [30],
    qualifiers = "w221dp-h221dp-small-notlong-notround-watch-xhdpi-keyshidden-nonav"
)

@Config(
    sdk = [30],
    qualifiers = "w221dp-h221dp-small-notlong-round-watch-xhdpi-keyshidden-nonav"
)

Hey,

I am trying to run Roborazzi snapshots in the pipeline which is configured on Jenkins. However, I am facing a few issues (or maybe this is something I misunderstood in the setup process and you will be able to guide me). There are two issues that I want to stress, which might be related:

Prerequisites:

• I am running the 1.2.0 version
• Have baseline stored in the project
• I am using the most simple Roborazzi Rule setup. RoborazziRule(onView(isRoot()))

1. Running verifyRoborazziDebug task in Jenkins.
   No matter what path I will declare in the captureRoboImage(path) for the snapshot using, it will always fail. However, this doesn't apply to the local gradle task run. I have a test that should be using a custom path to record and verify the snapshot state. It works locally, but in Jenkins, I always get this:

<...>MyTest > example1 FAILED
    java.lang.AssertionError: Roborazzi: /var/lib/jenkins/workspace/<...>/build/outputs/roborazzi/example1_compare.png is added.
    See compare image at /var/lib/jenkins/workspace/<...>/build/outputs/roborazzi/example1_compare.png
        at com.github.takahirom.roborazzi.RoborazziOptions$RoborazziCompareReporter$VerifyRoborazziCompareReporter.report(capture.kt:349)
        at com.github.takahirom.roborazzi.RoborazziKt.processOutputImageAndReport(Roborazzi.kt:666)
        at com.github.takahirom.roborazzi.RoborazziKt.saveLastImage(Roborazzi.kt:472)
        at com.github.takahirom.roborazzi.RoborazziKt.access$saveLastImage(Roborazzi.kt:1)
        at com.github.takahirom.roborazzi.RoborazziKt$captureAndroidView$3.invoke(Roborazzi.kt:432)
        at com.github.takahirom.roborazzi.RoborazziKt$captureAndroidView$3.invoke(Roborazzi.kt:431)
        at com.github.takahirom.roborazzi.RoborazziRule$apply$1.evaluate(RoborazziRule.kt:133)
        at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
        at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
        at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.robolectric.RobolectricTestRunner$HelperTestRunner$1.evaluate(RobolectricTestRunner.java:589)
        at org.robolectric.internal.SandboxTestRunner$2.lambda$evaluate$2(SandboxTestRunner.java:290)
        at org.robolectric.internal.bytecode.Sandbox.lambda$runOnMainThread$0(Sandbox.java:99)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

It wasn't working by setting outputDirectoryPath too. By the stacktrace it is clear that Roborazzi is using the DEFAULT_ROBORAZZI_OUTPUT_DIR_PATH for the snapshot verification.

A bit more context:
I have tried to force the snapshot to fail (with an outdated baseline) I got the *_compare.png generated in the path I declared in the captureRoboImage(path) method. So the outcome was as expected but logs were stating a different path to the *_capture.png image. When I update the baseline for the test to succeed, it had no *_compare.png image under the declared path. However, it still got me the top error.

2. Running any Roborazzi task in Jenkins.
   It doesn't work properly, since it always gets the Skipped even tho each time I change the test count or snapshotted view content or test steps/content. As I was investigating why that happens I made a conclusion, that if Gradle consistently marks tasks as up-to-date even though they aren't, it suggests that the tasks' inputs or outputs may not be configured correctly. Roborazzi needs to ensure that any files tasks rely on are properly declared as inputs, and any files or directories tasks generate are declared as outputs.

The only way to run the task is by adding the flag to the gradle command ./gradlew verifyRoborazzi --rerun-tasks.

It might be related to the first issue.

Skipping task ':testDebugUnitTest' as it is up-to-date.
Resolve mutations for :verifyRoborazziDebug (Thread[included builds,5,main]) started.
:verifyRoborazziDebug (Thread[included builds,5,main]) started.

> Task :verifyRoborazziDebug UP-TO-DATE
Skipping task ':verifyRoborazziDebug' as it has no actions.

p.s. if you will try to set up a Jenkins pipeline to test, don't forget to append to the start of the path variable the working directory path System.getProperty("user.dir"). This cost me some time to figure this out.

Thanks for the great library and for the support!

Currently, the test process is only generating images for comparison and not causing the test to fail when expected. To improve our testing process and make it more accurate, we should separate the compareRoborazziDebug and verifyRoborazziDebug Gradle tasks.

Proposed Changes:

• Create a new Gradle task called compareRoborazziDebug that will handle image comparison.
• Modify the existing Gradle task, verifyRoborazziDebug, to focus on verifying the results of the comparison and cause the test to fail if necessary.
• Update the documentation to reflect these changes and provide instructions for using the new tasks.

By separating these tasks, we can better understand the testing results and address any discrepancies more efficiently.

#23

We may want to pass parameters like resizeScale or pixelBitConfig.
I'm going to add some options as system properties. You can pass parameters like this: "./gradlew recordRoborazziDebug -PresizeScale=0.5"

Are there any parameters you'd like to set as default?

Thank you for building this library. I tried to generate a screenshot in my existing Android app project using Robolectric 4.10.3 by adding the following:

# libs.versions.toml
roborazzi = "1.3.0-alpha-4"
roborazzi-core = { group = "io.github.takahirom.roborazzi", name = "roborazzi", version.ref = "roborazzi" }
roborazzi-compose = { group = "io.github.takahirom.roborazzi", name = "roborazzi-compose", version.ref = "roborazzi" }
roborazzi-junitRule = { group = "io.github.takahirom.roborazzi", name = "roborazzi-junit-rule", version.ref = "roborazzi" }
[plugins]
roborazzi = { id = "io.github.takahirom.roborazzi", version.ref = "roborazzi" }

// build.gradle.kts
plugins {
    // ...
    alias(libs.plugins.roborazzi) apply false
}

// app/build.gradle.kts
plugins {
    // ...
    alias(libs.plugins.roborazzi)
}
dependencies {
    // ...
    testImplementation(libs.roborazzi.core)
    testImplementation(libs.roborazzi.compose)
    testImplementation(libs.roborazzi.junitRule)
}

// test/kotlin/com/myapp/RoborazziTest.kt
package com.myapp

import androidx.compose.material3.Text
import androidx.test.ext.junit.runners.AndroidJUnit4
import com.github.takahirom.roborazzi.captureRoboImage
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.annotation.GraphicsMode
import org.robolectric.annotation.GraphicsMode.Mode.NATIVE

@RunWith(AndroidJUnit4::class)
@GraphicsMode(NATIVE)
class RoborazziTest {
    @Test
    fun roborazziTest() {
        captureRoboImage {
            Text("Hello Compose!")
        }
    }
}

I then ran the test (along with all other tests the app already has - how to run only this one and still record the screenshot from Android Studio?) as follows:

./gradlew recordRoborazziDebug

But I then encounter the following error:

java.lang.RuntimeException: Unable to resolve activity for Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.myapp.debug/com.github.takahirom.roborazzi.RoborazziTransparentActivity } -- see https://github.com/robolectric/robolectric/pull/4736 for details
	at org.robolectric.android.internal.RoboMonitoringInstrumentation.startActivitySyncInternal(RoboMonitoringInstrumentation.java:98)
	at org.robolectric.android.internal.LocalActivityInvoker.startActivity(LocalActivityInvoker.java:35)
	at org.robolectric.android.internal.LocalActivityInvoker.startActivity(LocalActivityInvoker.java:40)
	at androidx.test.core.app.ActivityScenario.launchInternal(ActivityScenario.java:362)
	at androidx.test.core.app.ActivityScenario.launch(ActivityScenario.java:202)
	at com.github.takahirom.roborazzi.RoborazziComposeKt.captureRoboImage(RoborazziCompose.kt:29)
	at com.github.takahirom.roborazzi.RoborazziComposeKt.captureRoboImage(RoborazziCompose.kt:16)
	at com.github.takahirom.roborazzi.RoborazziComposeKt.captureRoboImage$default(RoborazziCompose.kt:11)
	at com.myapp.RoborazziTest.roborazziTest(RoborazziTest.kt:16)

However, I don't know what to make with the provided Robolectric PR link. What should I concretely do to make the task succeed and generate the screenshot? Currently there is no folder build/outputs/roborazzi generated.

For inspiration https://github.com/joreilly/Confetti/blob/de9c9cd1f309071a44f6879c637e895770573a81/androidTest/src/main/kotlin/dev/johnoreilly/confetti/screenshot/a11y/A11ySnapshotTransformer.kt#L17

Results in screenshots like https://github.com/joreilly/Confetti/blob/de9c9cd1f309071a44f6879c637e895770573a81/wearApp/snapshot/ConferenceHomeScreenTest/conferenceHomeScreenA11y_Colors_Material__reference.png

I'm considering changing the default behavior of RoborazziRule. As it stands, it is configured to take screenshots for each method by default, since the rule uses CaptureType.LastImage as a default value. However, this has proven to be very confusing and has led to the creation of several issues. What do you think about changing it to CaptureType.None and simply providing some context, such as the default output path?

I have numerous tests in my project. I would like to run Roborazzi tests using ./gradlew recordRoborazziDebug only if it is a Roborazzi test and use ./gradlew testDebugUnitTest for non-Roborazzi tests.
Although we have the --tests parameter, it cannot filter out non-Roborazzi tests, like using --excludeTests "*ScreenshotTest."
Is there any way to achieve this? I am concerned that we should not significantly affect the standard test task.

Existing screenshots before roborazzi

After roborazzi google/horologist#1234

Context: #4 (comment)

I think we can easily implement this by adding this parameter.
(result: ImageComparator.ComparisonResult) -> Boolean

The company that I work for is using Bazel as the build tool for our application and we are considering using Roborazzi for snapshot testing in the future. We've done some internal testing and it looks like the tool doesn't need the Gradle plugin to successfully run since you can pass in the environment flag. For instance in Gradle we could do:

tasks.withType<Test> {
  doFirst {
    systemProperties["roborazzi.test.record"] = "true"
  }
}

All tests that incorporate Roborazzi will then output the snapshots. (Obviously they will always do that but that's ok for us since that's our desired output).

Ideally we would build a custom rule on android_local_test (something like snapshot_local_test) where we pass in the custom environment variable for Roborazzi.

The challenge is then Bazel outputs everything into a build output directory (https://bazel.build/remote/output-directories). Is it possible to bake support for moving the snapshot tests to the root src directory after a successful rule has executed?

Currently, if you verify when the golden is not present:

java.lang.AssertionError: Roborazzi: [removed]/ForYouScreenLoading_foldable_compare.png is added.
See compare image at [removed]/ForYouScreenLoading_foldable_compare.png
	at com.github.takahirom.roborazzi.RoborazziOptions$RoborazziCompareReporter$VerifyRoborazziCompareReporter.report(capture.kt:349)
	at com.github.takahirom.roborazzi.RoborazziKt.processOutputImageAndReport(Roborazzi.kt:650)
	at com.github.takahirom.roborazzi.RoborazziKt.access$processOutputImageAndReport(Roborazzi.kt:1)
	at com.github.takahirom.roborazzi.RoborazziKt$captureRoboImage$2.invoke(Roborazzi.kt:274)
	at com.github.takahirom.roborazzi.RoborazziKt$captureRoboImage$2.invoke(Roborazzi.kt:267)
	at com.github.takahirom.roborazzi.RoborazziKt.capture(Roborazzi.kt:708)
	at com.github.takahirom.roborazzi.RoborazziKt.captureRoboImage(Roborazzi.kt:267)
	at com.github.takahirom.roborazzi.RoborazziKt.captureRoboImage(Roborazzi.kt:259)
	at com.github.takahirom.roborazzi.RoborazziKt.captureRoboImage$default(Roborazzi.kt:254)

The "Error" is not "the png is added". It should say that the original file doesn't exist or that there are differences between the images.

I propose to initiate work on a roborazzi-desktop module, which will be developed using JVM for proof-of-concept (PoC). Following this, I suggest moving the core logic from roborazzi to roborazzi-core and then implementing roborazzi-desktop. I believe this would be an effective approach for adding Desktop support to Roborazzi.

When I upgrade to 1.6.0-alpha-2, I get the following error:

Failed to notify project evaluation listener.
   > Cannot change attributes of dependency configuration ':image-loader:iosArm64ApiElements' after it has been resolved
   > Cannot change attributes of dependency configuration ':image-loader:iosX64ApiElements' after it has been resolved
   > Cannot change attributes of dependency configuration ':image-loader:iosSimulatorArm64ApiElements' after it has been resolved
   > Cannot change attributes of dependency configuration ':image-loader:macosX64ApiElements' after it has been resolved
   > Cannot change attributes of dependency configuration ':image-loader:macosArm64ApiElements' after it has been resolved
   > Cannot change attributes of dependency configuration ':image-loader:jsMainApiDependenciesMetadata' after it has been resolved
   > Cannot change attributes of dependency configuration ':image-loader:jsMainImplementationDependenciesMetadata' after it has been resolved
   > Cannot change attributes of dependency configuration ':image-loader:jsMainCompileOnlyDependenciesMetadata' after it has been resolved
   > Cannot change attributes of dependency configuration ':image-loader:jsMainIntransitiveDependenciesMetadata' after it has been resolved
   > Cannot change attributes of dependency configuration ':image-loader:jsTestApiDependenciesMetadata' after it has been resolved
   > Cannot change attributes of dependency configuration ':image-loader:jsTestImplementationDependenciesMetadata' after it has been resolved
   > Cannot change attributes of dependency configuration ':image-loader:jsTestCompileOnlyDependenciesMetadata' after it has been resolved
   > Cannot change attributes of dependency configuration ':image-loader:jsTestIntransitiveDependenciesMetadata' after it has been resolved

We can enhance our documentation by providing a sample GitHub Actions workflow in the README file. This will help users to quickly understand and integrate our solution into their projects with ease.

The Gradle plugin for Roborazzi 1.0.0 isn't published:

> Could not resolve all files for configuration ':classpath'.
   > Could not find io.github.takahirom.roborazzi:roborazzi-gradle-plugin:1.0.0.
     Searched in the following locations:
       - https://dl.google.com/dl/android/maven2/io/github/takahirom/roborazzi/roborazzi-gradle-plugin/1.0.0/roborazzi-gradle-plugin-1.0.0.pom
       - https://repo.maven.apache.org/maven2/io/github/takahirom/roborazzi/roborazzi-gradle-plugin/1.0.0/roborazzi-gradle-plugin-1.0.0.pom
       - https://plugins.gradle.org/m2/io/github/takahirom/roborazzi/roborazzi-gradle-plugin/1.0.0/roborazzi-gradle-plugin-1.0.0.pom
     Required by:
         project :

Hi, thanks for build this library. I did the configuration as same as example.

@RunWith(AndroidJUnit4::class)
@GraphicsMode(GraphicsMode.Mode.NATIVE)
class ComposeScreenShotTest {

    @get:Rule
    val composeTestRule = createAndroidComposeRule<ComponentActivity>()

    @get:Rule
    val roborazziRule = RoborazziRule(
        composeRule = composeTestRule,
        captureRoot = composeTestRule.onRoot(),
        options = RoborazziRule.Options(
            outputDirectoryPath = "src/androidUnitTest/snapshots/images",
        ),
    )

    @Test
    fun loadImage() {
        composeTestRule.setContent {
            Spacer(
                Modifier.size(100.dp).background(Color.Red),
            )
        }
    }
}

But when I run ./gradlew recordRoborazziDebug, generated images is like this:

My environment:

• Gradle: 8.2
• AGP: 8.0.2
• Kotlin: 1.8.20
• Compose-MultiPlatform: 1.4.1
• Roborazzi: 1.4.0-alpha-2
• Robolectric: 4.10.3

Since Robolectric cannot render shadows, it is pretty useful to provide a background color to the activity launched via ActivityScenario to better distinguish the "borders", e.g. in CardViews.

AndroidUiTestingUtils offers that possibility like this (same for composables), for instance:

@get:Rule
val activityScenarioForViewRule =
        ActivityScenarioForViewRule(
            config = ViewConfigItem(...),
            backgroundColor = Color.TRANSPARENT,
        )

When used with Roborazzi, if backgroundColor = Color.TRANSPARENT, the background color is eventually black (I believe, it is might be set to null?). It works with other colours though

Moreover, I believe it is related to Roborazzi and not Robolectric, since if I use the following method compareSnapshot, instead of captureRoboImage(), the background is actually transparent...

fun compareSnapshot(view: View, name: String) {
    val image = view.drawToBitmap()
    val path = System.getProperty("user.dir")
    val file = File("$path/src/test", "$name.png")

    try {
        val bos = ByteArrayOutputStream()
        val out = FileOutputStream(file)
        out.write(bos.toByteArray());
        image.compress(Bitmap.CompressFormat.PNG, 100, out)
        out.flush()
        out.close()
    } catch (e: Exception) {
        e.printStackTrace()
    }
}

Current result with TRANSPARENT backgroundColor:

With captureRoboImage()

With compareSnapshot():

In the environment without the Compose and by simply capturing the robo image, DefaultFineNameGeneration crashes when trying to search for the test method name.

onView(isRoot())
                .captureRoboImage()

The stacktrace is:

java.lang.NoClassDefFoundError: androidx/compose/ui/test/SemanticsNodeInteraction
	at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
	at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3166)
	at java.base/java.lang.Class.getMethodsRecursive(Class.java:3307)
	at java.base/java.lang.Class.getMethod0(Class.java:3293)
	at java.base/java.lang.Class.getMethod(Class.java:2106)
	at com.github.takahirom.roborazzi.DefaultFileNameGenerator.generateName(DefaultFileNameGenerator.kt:26)
	at com.github.takahirom.roborazzi.DefaultFileNameGenerator.generateFilePath(DefaultFileNameGenerator.kt:11)
	at com.github.takahirom.roborazzi.RoborazziKt.captureRoboImage$default(Roborazzi.kt:67)

Not sure why the compose is considered here, but it crashes since the NoClassDefFoundError is Throwable and not the Exception.

1.1.0 version

I tried it and Roborazzi apparently works properly with Showkase.