Giter VIP home page Giter VIP logo

swift-snapshot-testing's Introduction

📸 SnapshotTesting

CI Slack

Delightful Swift snapshot testing.

Usage

Once installed, no additional configuration is required. You can import the SnapshotTesting module and call the assertSnapshot function.

import SnapshotTesting
import XCTest

class MyViewControllerTests: XCTestCase {
  func testMyViewController() {
    let vc = MyViewController()

    assertSnapshot(of: vc, as: .image)
  }
}

When an assertion first runs, a snapshot is automatically recorded to disk and the test will fail, printing out the file path of any newly-recorded reference.

❌ failed - No reference was found on disk. Automatically recorded snapshot: …

open "…/MyAppTests/__Snapshots__/MyViewControllerTests/testMyViewController.png"

Re-run "testMyViewController" to test against the newly-recorded snapshot.

Repeat test runs will load this reference and compare it with the runtime value. If they don't match, the test will fail and describe the difference. Failures can be inspected from Xcode's Report Navigator or by inspecting the file URLs of the failure.

You can record a new reference by setting the record parameter to true on the assertion or setting isRecording globally.

assertSnapshot(of: vc, as: .image, record: true)

// or globally

isRecording = true
assertSnapshot(of: vc, as: .image)

Snapshot Anything

While most snapshot testing libraries in the Swift community are limited to UIImages of UIViews, SnapshotTesting can work with any format of any value on any Swift platform!

The assertSnapshot function accepts a value and any snapshot strategy that value supports. This means that a view or view controller can be tested against an image representation and against a textual representation of its properties and subview hierarchy.

assertSnapshot(of: vc, as: .image)
assertSnapshot(of: vc, as: .recursiveDescription)

View testing is highly configurable. You can override trait collections (for specific size classes and content size categories) and generate device-agnostic snapshots, all from a single simulator.

assertSnapshot(of: vc, as: .image(on: .iPhoneSe))
assertSnapshot(of: vc, as: .recursiveDescription(on: .iPhoneSe))

assertSnapshot(of: vc, as: .image(on: .iPhoneSe(.landscape)))
assertSnapshot(of: vc, as: .recursiveDescription(on: .iPhoneSe(.landscape)))

assertSnapshot(of: vc, as: .image(on: .iPhoneX))
assertSnapshot(of: vc, as: .recursiveDescription(on: .iPhoneX))

assertSnapshot(of: vc, as: .image(on: .iPadMini(.portrait)))
assertSnapshot(of: vc, as: .recursiveDescription(on: .iPadMini(.portrait)))

Warning Snapshots must be compared using the exact same simulator that originally took the reference to avoid discrepancies between images.

Better yet, SnapshotTesting isn't limited to views and view controllers! There are a number of available snapshot strategies to choose from.

For example, you can snapshot test URL requests (e.g., those that your API client prepares).

assertSnapshot(of: urlRequest, as: .raw)
// POST http://localhost:8080/account
// Cookie: pf_session={"userId":"1"}
//
// email=blob%40pointfree.co&name=Blob

And you can snapshot test Encodable values against their JSON and property list representations.

assertSnapshot(of: user, as: .json)
// {
//   "bio" : "Blobbed around the world.",
//   "id" : 1,
//   "name" : "Blobby"
// }

assertSnapshot(of: user, as: .plist)
// <?xml version="1.0" encoding="UTF-8"?>
// <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
//  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
// <plist version="1.0">
// <dict>
//   <key>bio</key>
//   <string>Blobbed around the world.</string>
//   <key>id</key>
//   <integer>1</integer>
//   <key>name</key>
//   <string>Blobby</string>
// </dict>
// </plist>

In fact, any value can be snapshot-tested by default using its mirror!

assertSnapshot(of: user, as: .dump)
// ▿ User
//   - bio: "Blobbed around the world."
//   - id: 1
//   - name: "Blobby"

If your data can be represented as an image, text, or data, you can write a snapshot test for it!

Documentation

The latest documentation is available here.

Installation

Xcode

Warning By default, Xcode will try to add the SnapshotTesting package to your project's main application/framework target. Please ensure that SnapshotTesting is added to a test target instead, as documented in the last step, below.

  1. From the File menu, navigate through Swift Packages and select Add Package Dependency….
  2. Enter package repository URL: https://github.com/pointfreeco/swift-snapshot-testing.
  3. Confirm the version and let Xcode resolve the package.
  4. On the final dialog, update SnapshotTesting's Add to Target column to a test target that will contain snapshot tests (if you have more than one test target, you can later add SnapshotTesting to them by manually linking the library in its build phase).

Swift Package Manager

If you want to use SnapshotTesting in any other project that uses SwiftPM, add the package as a dependency in Package.swift:

dependencies: [
  .package(
    url: "https://github.com/pointfreeco/swift-snapshot-testing",
    from: "1.12.0"
  ),
]

Next, add SnapshotTesting as a dependency of your test target:

targets: [
  .target(name: "MyApp"),
  .testTarget(
    name: "MyAppTests",
    dependencies: [
      "MyApp",
      .product(name: "SnapshotTesting", package: "swift-snapshot-testing"),
    ]
  )
]

Features

  • Dozens of snapshot strategies. Snapshot testing isn't just for UIViews and CALayers. Write snapshots against any value.
  • Write your own snapshot strategies. If you can convert it to an image, string, data, or your own diffable format, you can snapshot test it! Build your own snapshot strategies from scratch or transform existing ones.
  • No configuration required. Don't fuss with scheme settings and environment variables. Snapshots are automatically saved alongside your tests.
  • More hands-off. New snapshots are recorded whether isRecording mode is true or not.
  • Subclass-free. Assert from any XCTest case or Quick spec.
  • Device-agnostic snapshots. Render views and view controllers for specific devices and trait collections from a single simulator.
  • First-class Xcode support. Image differences are captured as XCTest attachments. Text differences are rendered in inline error messages.
  • Supports any platform that supports Swift. Write snapshot tests for iOS, Linux, macOS, and tvOS.
  • SceneKit, SpriteKit, and WebKit support. Most snapshot testing libraries don't support these view subclasses.
  • Codable support. Snapshot encodable data structures into their JSON and property list representations.
  • Custom diff tool integration. Configure failure messages to print diff commands for Kaleidoscope (or your diff tool of choice).
    SnapshotTesting.diffTool = "ksdiff"

Plug-ins

Have you written your own SnapshotTesting plug-in? Add it here and submit a pull request!

Related Tools

  • iOSSnapshotTestCase helped introduce screen shot testing to a broad audience in the iOS community. Experience with it inspired the creation of this library.

  • Jest brought generalized snapshot testing to the JavaScript community with a polished user experience. Several features of this library (diffing, automatically capturing new snapshots) were directly influenced.

Learn More

SnapshotTesting was designed with witness-oriented programming.

This concept (and more) are explored thoroughly in a series of episodes on Point-Free, a video series exploring functional programming and Swift hosted by Brandon Williams and Stephen Celis.

Witness-oriented programming and the design of this library was explored in the following Point-Free episodes:

video poster image

License

This library is released under the MIT license. See LICENSE for details.

swift-snapshot-testing's People

Contributors

ejensen avatar f-meloni avatar ferranpujolcamins avatar finestructure avatar freak4pc avatar imvm avatar luisrecuenco avatar mackoj avatar maxdesiatov avatar mbrandonw avatar mihai8804858 avatar mika5652 avatar moritzsternemann avatar mr-v avatar mstultz avatar nachosoto avatar natemann avatar nuno-vieira avatar reez avatar regexident avatar rjchatfield avatar schwmi avatar sherlouk avatar slashmo avatar stephencelis avatar teameh avatar thedavidharris avatar tinder-maxwellelliott avatar valeriyvan avatar xavierlowmiller avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

swift-snapshot-testing's Issues

try a type-erased struct instead of Diffable protocol

protocols that only have Self and no associated types can be represented by a simple struct:

struct Semigroup<A> {
  let op: (A, A) -> A
}

let additiveInt = Semigroup<Int> { $0 + $1 }
let orBool = Semigroup<Bool> { $0 || $1 }
let andBool = Semigroup<Bool> { $0 && $1 }

we can use this idea for Diffable and would mean a single type could have multiple diffable representations.

I think then Snapshot could have an init(diffable: Diffable<A>) initializer, or maybe even be merged with Diffable protocol

Timeouts on WKWebView image snapshot

Hi, one more issue from me 😇 It looks like WKWebView is never performing image snapshot when there is no content loaded. I have the case where one of my views contains WKWebView that have no content initially. If I've debugged that correctly image snapshot on web view waits until the page finishes loading. But when there is no page loading it never occurs resulting in a timeout.
I am wondering if making it possible to take a snapshot when there is no content in web view is a good solution here.
Workaround for this is to load any file from disk into, even empty one.

Improve view strategy

Right now we have separate UIView/NSView and WKWebView strategies, but really we should have a single UIView/NSView strategy that traverses the hierarchy and creates snapshots of views that aren't captured with layer.render(in:), including:

  • GLKView
  • SCNView
  • SKView
  • WKWebView

Rename again?

I'm thinking maybe swift-snapshot-testing (import SnapshotTesting) makes more sense.

Generic parameter 'Format' could not be inferred

Hi,
I'm trying the following test in my project:

//
//  ManualEntriesCustomerFunnel.swift
//

import XCTest
import SnapshotTesting
@testable import pos

class AlertControllerMessages: XCTestCase {

    func testAlertController() {
        let vc = GeneralAlert.errorWith(body: "Hi there", buttonText: "Cancel", cancelAction: nil)
        assertSnapshot(matching: vc, as: .image)
    }

}

The compiler its giving me an error Generic parameter 'Format' could not be inferred for the .image parameter.
Do I need to implement something else?

Track outdated snapshots

We should track outdated snapshots and potentially fail if any exist. We can add a quick confirmation helper to prune outdated snapshots.

Crash on image snapshot of UIViewController without autoresizing mask constraints

I have found crash in one of my apps when running image snapshot of UIViewController. In this specific case ViewController was ment to use as child and its view have translatesAutoresizingMasksIntoConstraints property set to false. When trying to record snapshot on such ViewController snapshot was not able to produce png while somehow passing size guards (it looks like its size is 0,0).
Here is a stack trace from recording:

thread #1, queue = 'com.apple.main-thread', stop reason = Fatal error: Unexpectedly found nil while unwrapping an Optional value
    frame #0: 0x000000010f5d5960 libswiftCore.dylib`_swift_runtime_on_report
    frame #1: 0x000000010f627331 libswiftCore.dylib`_swift_stdlib_reportFatalError + 113
    frame #2: 0x000000010f32d57a libswiftCore.dylib`function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> () to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out ()), Argument Types : [@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> ()]> of generic specialization <()> of Swift.StaticString.withUTF8Buffer<A>((Swift.UnsafeBufferPointer<Swift.UInt8>) -> A) -> A + 58
    frame #3: 0x000000010f5b573e libswiftCore.dylib`partial apply forwarder for closure #2 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 110
    frame #4: 0x000000010f32d57a libswiftCore.dylib`function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> () to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out ()), Argument Types : [@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> ()]> of generic specialization <()> of Swift.StaticString.withUTF8Buffer<A>((Swift.UnsafeBufferPointer<Swift.UInt8>) -> A) -> A + 58
    frame #5: 0x000000010f4f76e9 libswiftCore.dylib`function signature specialization <Arg[2] = Dead, Arg[3] = Dead> of Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 105
    frame #6: 0x000000010f32d053 libswiftCore.dylib`Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 19
  * frame #7: 0x0000000129a1298a SnapshotTesting`closure #1 in static Snapshotting<>.image($0=0x0000600000fb4070) at UIImage.swift:25
    frame #8: 0x0000000129a129cf SnapshotTesting`thunk for @escaping @callee_guaranteed (@guaranteed UIImage) -> (@owned Data) at <compiler-generated>:0
    frame #9: 0x0000000129a12a31 SnapshotTesting`partial apply for thunk for @escaping @callee_guaranteed (@guaranteed UIImage) -> (@owned Data) at <compiler-generated>:0
    frame #10: 0x00000001299b81cd SnapshotTesting`verifySnapshot<A, B>(value=0x0000000129999230 AppTests`reabstraction thunk helper from @callee_guaranteed () -> (@owned __C.UIViewController, @error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out __C.UIViewController, @error @owned Swift.Error)partial apply forwarder with unmangled suffix ".21" at <compiler-generated>, snapshotting=SnapshotTesting.Snapshotting<UIKit.UIViewController, UIKit.UIImage> @ 0x00007ffee24a66e8, name=nil, recording=false, timeout=5, file="AppTests/DashboardViewTests.swift", testName="testViewe()", line=30) at AssertSnapshot.swift:122
    frame #11: 0x00000001299b4f08 SnapshotTesting`assertSnapshot<A, B>(value=0x0000000129999230 AppTests`reabstraction thunk helper from @callee_guaranteed () -> (@owned __C.UIViewController, @error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out __C.UIViewController, @error @owned Swift.Error)partial apply forwarder with unmangled suffix ".21" at <compiler-generated>, snapshotting=SnapshotTesting.Snapshotting<UIKit.UIViewController, UIKit.UIImage> @ 0x00007ffee24a66e8, name=nil, recording=false, timeout=5, file="AppTests/DashboardViewTests.swift", testName="testViewe()", line=30) at AssertSnapshot.swift:34
    frame #12: 0x0000000129998a9c AppTests`DashboardViewTests.testViewe(self=0x00006000024f5aa0) at DashboardViewTests.swift:30
    frame #13: 0x0000000129999304 AppTests`@objc DashboardViewTests.testViewe() at <compiler-generated>:0
    frame #14: 0x000000010ee8c03c CoreFoundation`__invoking___ + 140
    frame #15: 0x000000010ee894d5 CoreFoundation`-[NSInvocation invoke] + 325
    frame #16: 0x0000000127e7b018 XCTest`__24-[XCTestCase invokeTest]_block_invoke.196 + 78
    frame #17: 0x0000000127ed2c74 XCTest`-[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
    frame #18: 0x0000000127ed2b91 XCTest`-[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
    frame #19: 0x0000000127e7acd9 XCTest`__24-[XCTestCase invokeTest]_block_invoke + 848
    frame #20: 0x0000000127ed8b7e XCTest`-[XCUITestContext performInScope:] + 248
    frame #21: 0x0000000127e7a8ce XCTest`-[XCTestCase testContextPerformInScope:] + 98
    frame #22: 0x0000000127e7a97c XCTest`-[XCTestCase invokeTest] + 137
    frame #23: 0x0000000127e7c4b7 XCTest`__26-[XCTestCase performTest:]_block_invoke_2 + 43
    frame #24: 0x0000000127ed2c74 XCTest`-[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
    frame #25: 0x0000000127ed2b91 XCTest`-[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
    frame #26: 0x0000000127e7c3ce XCTest`__26-[XCTestCase performTest:]_block_invoke.330 + 88
    frame #27: 0x0000000127ee344b XCTest`+[XCTContext runInContextForTestCase:block:] + 225
    frame #28: 0x0000000127e7bafd XCTest`-[XCTestCase performTest:] + 675
    frame #29: 0x0000000127ebf1a2 XCTest`-[XCTest runTest] + 57
    frame #30: 0x0000000127e76ccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
    frame #31: 0x0000000127e764a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
    frame #32: 0x0000000127e76766 XCTest`-[XCTestSuite performTest:] + 296
    frame #33: 0x0000000127ebf1a2 XCTest`-[XCTest runTest] + 57
    frame #34: 0x0000000127e76ccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
    frame #35: 0x0000000127e764a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
    frame #36: 0x0000000127e76766 XCTest`-[XCTestSuite performTest:] + 296
    frame #37: 0x0000000127ebf1a2 XCTest`-[XCTest runTest] + 57
    frame #38: 0x0000000127e76ccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
    frame #39: 0x0000000127e764a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
    frame #40: 0x0000000127e76766 XCTest`-[XCTestSuite performTest:] + 296
    frame #41: 0x0000000127ebf1a2 XCTest`-[XCTest runTest] + 57
    frame #42: 0x0000000127eeee86 XCTest`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 171
    frame #43: 0x0000000127eeefa7 XCTest`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke.80 + 68
    frame #44: 0x0000000127e8ebc1 XCTest`-[XCTestObservationCenter _observeTestExecutionForBlock:] + 585
    frame #45: 0x0000000127eeebfa XCTest`-[XCTTestRunSession runTestsAndReturnError:] + 623
    frame #46: 0x0000000127e5b6b6 XCTest`-[XCTestDriver runTestsAndReturnError:] + 422
    frame #47: 0x0000000127edf9cd XCTest`_XCTestMain + 1478
    frame #48: 0x000000010d851bb8 libXCTestBundleInject.dylib`__RunTests_block_invoke_2 + 13
    frame #49: 0x000000010edea62c CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    frame #50: 0x000000010ede9de0 CoreFoundation`__CFRunLoopDoBlocks + 336
    frame #51: 0x000000010ede4654 CoreFoundation`__CFRunLoopRun + 1284
    frame #52: 0x000000010ede3e11 CoreFoundation`CFRunLoopRunSpecific + 625
    frame #53: 0x00000001146a71dd GraphicsServices`GSEventRunModal + 62
    frame #54: 0x000000011a3cb81d UIKitCore`UIApplicationMain + 140
    frame #55: 0x000000010d757b27 AppApp`main at AppDelegate.swift:14
    frame #56: 0x0000000111285575 libdyld.dylib`start + 1

and from comparation with existing snapshot:

thread #1, queue = 'com.apple.main-thread', stop reason = Fatal error: Unexpectedly found nil while unwrapping an Optional value
    frame #0: 0x000000010fffd960 libswiftCore.dylib`_swift_runtime_on_report
    frame #1: 0x000000011004f331 libswiftCore.dylib`_swift_stdlib_reportFatalError + 113
    frame #2: 0x000000010fd5557a libswiftCore.dylib`function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> () to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out ()), Argument Types : [@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> ()]> of generic specialization <()> of Swift.StaticString.withUTF8Buffer<A>((Swift.UnsafeBufferPointer<Swift.UInt8>) -> A) -> A + 58
    frame #3: 0x000000010ffdd73e libswiftCore.dylib`partial apply forwarder for closure #2 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 110
    frame #4: 0x000000010fd5557a libswiftCore.dylib`function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> () to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out ()), Argument Types : [@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> ()]> of generic specialization <()> of Swift.StaticString.withUTF8Buffer<A>((Swift.UnsafeBufferPointer<Swift.UInt8>) -> A) -> A + 58
    frame #5: 0x000000010ff1f6e9 libswiftCore.dylib`function signature specialization <Arg[2] = Dead, Arg[3] = Dead> of Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 105
    frame #6: 0x000000010fd55053 libswiftCore.dylib`Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 19
  * frame #7: 0x000000012a43d77e SnapshotTesting`diff(old=0x0000600003dc4620, new=0x0000600003dc45b0) at UIImage.swift:99
    frame #8: 0x000000012a43ac93 SnapshotTesting`closure #3 in static Snapshotting<>.image(old=0x0000600003dc4620, new=0x0000600003dc45b0, precision=1) at UIImage.swift:29
    frame #9: 0x000000012a43b15f SnapshotTesting`partial apply for closure #3 in static Snapshotting<>.image(precision:) at <compiler-generated>:0
    frame #10: 0x000000012a43ddb2 SnapshotTesting`thunk for @escaping @callee_guaranteed (@guaranteed UIImage, @guaranteed UIImage) -> (@owned (String, [Attachment])?) at <compiler-generated>:0
    frame #11: 0x000000012a43de11 SnapshotTesting`partial apply for thunk for @escaping @callee_guaranteed (@guaranteed UIImage, @guaranteed UIImage) -> (@owned (String, [Attachment])?) at <compiler-generated>:0
    frame #12: 0x000000012a3de994 SnapshotTesting`verifySnapshot<A, B>(value=0x000000012a3c0d60 AppTests`partial apply forwarder for reabstraction thunk helper from @callee_guaranteed () -> (@owned __C.UIViewController, @error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out __C.UIViewController, @error @owned Swift.Error) at <compiler-generated>, snapshotting=SnapshotTesting.Snapshotting<UIKit.UIViewController, UIKit.UIImage> @ 0x00007ffee1a7e648, name=nil, recording=false, timeout=5, file="AppTests/DashboardViewTests.swift", testName="testView()", line=25) at AssertSnapshot.swift:129
    frame #13: 0x000000012a3dcf08 SnapshotTesting`assertSnapshot<A, B>(value=0x000000012a3c0d60 AppTests`partial apply forwarder for reabstraction thunk helper from @callee_guaranteed () -> (@owned __C.UIViewController, @error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out __C.UIViewController, @error @owned Swift.Error) at <compiler-generated>, snapshotting=SnapshotTesting.Snapshotting<UIKit.UIViewController, UIKit.UIImage> @ 0x00007ffee1a7e648, name=nil, recording=false, timeout=5, file="AppTests/DashboardViewTests.swift", testName="testView()", line=25) at AssertSnapshot.swift:34
    frame #14: 0x000000012a3c0455 AppTests`DashboardViewTests.testView(self=0x0000600001680400) at DashboardViewTests.swift:25
    frame #15: 0x000000012a3c1314 AppTests`@objc DashboardViewTests.testView() at <compiler-generated>:0
    frame #16: 0x000000010f8b403c CoreFoundation`__invoking___ + 140
    frame #17: 0x000000010f8b14d5 CoreFoundation`-[NSInvocation invoke] + 325
    frame #18: 0x00000001288a2018 XCTest`__24-[XCTestCase invokeTest]_block_invoke.196 + 78
    frame #19: 0x00000001288f9c74 XCTest`-[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
    frame #20: 0x00000001288f9b91 XCTest`-[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
    frame #21: 0x00000001288a1cd9 XCTest`__24-[XCTestCase invokeTest]_block_invoke + 848
    frame #22: 0x00000001288ffb7e XCTest`-[XCUITestContext performInScope:] + 248
    frame #23: 0x00000001288a18ce XCTest`-[XCTestCase testContextPerformInScope:] + 98
    frame #24: 0x00000001288a197c XCTest`-[XCTestCase invokeTest] + 137
    frame #25: 0x00000001288a34b7 XCTest`__26-[XCTestCase performTest:]_block_invoke_2 + 43
    frame #26: 0x00000001288f9c74 XCTest`-[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
    frame #27: 0x00000001288f9b91 XCTest`-[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
    frame #28: 0x00000001288a33ce XCTest`__26-[XCTestCase performTest:]_block_invoke.330 + 88
    frame #29: 0x000000012890a44b XCTest`+[XCTContext runInContextForTestCase:block:] + 225
    frame #30: 0x00000001288a2afd XCTest`-[XCTestCase performTest:] + 675
    frame #31: 0x00000001288e61a2 XCTest`-[XCTest runTest] + 57
    frame #32: 0x000000012889dccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
    frame #33: 0x000000012889d4a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
    frame #34: 0x000000012889d766 XCTest`-[XCTestSuite performTest:] + 296
    frame #35: 0x00000001288e61a2 XCTest`-[XCTest runTest] + 57
    frame #36: 0x000000012889dccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
    frame #37: 0x000000012889d4a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
    frame #38: 0x000000012889d766 XCTest`-[XCTestSuite performTest:] + 296
    frame #39: 0x00000001288e61a2 XCTest`-[XCTest runTest] + 57
    frame #40: 0x000000012889dccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
    frame #41: 0x000000012889d4a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
    frame #42: 0x000000012889d766 XCTest`-[XCTestSuite performTest:] + 296
    frame #43: 0x00000001288e61a2 XCTest`-[XCTest runTest] + 57
    frame #44: 0x0000000128915e86 XCTest`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 171
    frame #45: 0x0000000128915fa7 XCTest`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke.80 + 68
    frame #46: 0x00000001288b5bc1 XCTest`-[XCTestObservationCenter _observeTestExecutionForBlock:] + 585
    frame #47: 0x0000000128915bfa XCTest`-[XCTTestRunSession runTestsAndReturnError:] + 623
    frame #48: 0x00000001288826b6 XCTest`-[XCTestDriver runTestsAndReturnError:] + 422
    frame #49: 0x00000001289069cd XCTest`_XCTestMain + 1478
    frame #50: 0x000000010e279bb8 libXCTestBundleInject.dylib`__RunTests_block_invoke_2 + 13
    frame #51: 0x000000010f81262c CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    frame #52: 0x000000010f811de0 CoreFoundation`__CFRunLoopDoBlocks + 336
    frame #53: 0x000000010f80c654 CoreFoundation`__CFRunLoopRun + 1284
    frame #54: 0x000000010f80be11 CoreFoundation`CFRunLoopRunSpecific + 625
    frame #55: 0x00000001138221dd GraphicsServices`GSEventRunModal + 62
    frame #56: 0x0000000118cba81d UIKitCore`UIApplicationMain + 140
    frame #57: 0x000000010e17fb27 AppApp`main at AppDelegate.swift:14
    frame #58: 0x0000000111cad575 libdyld.dylib`start + 1

How to conform to DefaultSnapshottable

Hey there,

First off, this library looks awesome and I really enjoyed reading the article you posted about it.

I was starting to integrate this into a project of mine and was intrigued by applying snapshots to instance of arbitrary types—this seems to be one advantage of this lib over FBSnapshotTestCase. I quickly found out that my types would need to conform to DefaultSnapshottable in order to assert snapshot matches for them. I'm fine implementing conformance via extensions in my test target, but I've yet to figure out the intent of the associated types and how I might implement conformance.

Let's say I wanted to test NSAttributedString, since in my case my library is rendering an NSAttributedString with a lot of custom attributes. Any advice on how I would declare conformance to DefaultSnapshottable for NSAttributedString? Specifically, not sure what the associatedtype Format should be

Weird reference size when running on travis

My test run fine locally, but when I try to run it on Travis ci I got failed test with an error like this

Expected snapshot@(320.0, 44.0) to match reference@(213.33333333333334, 29.333333333333332)

I don't know why reference size showing weird resolution like 213.33333

Anyone experienced this before?

The file "Test.swift" doesn’t exist

Can not make SnapshotTesting work.
When I use assertSnapshot(matching: view, as: .image) I get this message "The file "failed "Test.swift" doesn’t exist"

What I can do ?

Question: Can the image comparision take into consideration some tolerance?

This is especially the case with FBSnapshot library which we use. We will like to switch to this one for recursive description text diff but might retain some image comparision. In that case we might need some tolerance for check. Does the library support it at the moment. Sorry, didnt have time to dig into the code or try out yet.

Crash when used in test target with asynchronous tests

As mentioned here. We have been trying to introduce the library to our project but are getting crashes in test targets which have asynchronous tests.

For example:

  • Using this library in a test target with tests that use Nimble's toEventually() async testing they crash in Nimble's code to manage the runloop
  • With those tests disabled, but with other async tests, the test suite completes, but when the app is shutting down after testing (in UIApplicationMain) we get a crash trying which appears to be in a call to objc_msgSend.

I enabled Zombie object diagnostics and in both of the above situations just before the crash I see this in the console:

*** -[CALayer setNeedsLayout]: message sent to deallocated instance 0x600000582da0

I'm sorry I can't provide more useful info right now. I will see if I can create a small project which reproduces the issue.

failed - Recorded snapshot: …

I have added the library using Carthage and referenced it in the unit test target. My first test looks like this

func testCreateChatRequestWithJustText() {
    let group = GroupEntity(groupId: "g1id", title: "G1")

    let normalRequest = ThreadCreateRequest(group: group, title: nil, notificationAlert: "U: text", encryptionSecret: "abcde", messageBody: "text", isAlert: false)
    if #available(iOS 11.0, *) {
        assertSnapshot(matching: normalRequest, as: .json)
    }
}

Running it fails with

failed - Recorded snapshot: …

"/Users/igorkulman/Projects/XXX/iOS/Core/CoreTests/Encodable/__Snapshots__/RequestsEncodingTests/testCreateChatRequestWithJustText.1.json"

If I add record = true it does not help, I get the same result.

Dictionary snapshots

Dictionary snapshotting is not deterministic. Let's consider the following snapshot.

let dictionary = ["first": 1, "second": 2, "third": 3]
assertSnapshot(matching: dictionary)

Each time the snapshot is made, the snapshot result is different.

▿ 3 key/value pairs
  ▿ (2 elements)
    - key: "third"
    - value: 3
  ▿ (2 elements)
    - key: "first"
    - value: 1
  ▿ (2 elements)
    - key: "second"
    - value: 2
▿ 3 key/value pairs
  ▿ (2 elements)
    - key: "first"
    - value: 1
  ▿ (2 elements)
    - key: "third"
    - value: 3
  ▿ (2 elements)
    - key: "second"
    - value: 2

Is there any way to handle this scenario correctly? Should we have a strategy for Dictionary<Comparable, T> so the snapshot is captured by sorting the keys in the first place?

Thanks

Group helper: assertSnapshots(matching:as:)

Would be nice to be able to assert on multiple snapshot strategies at once, as we do with a helper here here:

https://github.com/pointfreeco/pointfreeco/blob/8feb17812b2f076ab4996450ee85583ec77f6f54/Tests/PointFreeTests/AboutTests.swift#L20-L26

With it defined, we can also make an extension for some common things.

extension Dictionary where Key == String, Value == Snapshotting<UIViewController, UIImage> {
  static let allDevices: Dictionary = [
    "iphone-se-portrait": .image(on: .iPhoneSe(.portrait)),
    // …
  ]
}

// …

assertSnapshots(matching: vc, as: .allDevices)

Dealing With Display Scales

Question:

How do you deal with size classes on different devices?

Elaborate explanation:

We'd like to generate snapshots of views in different size classes.
To do this, we pass a UITraitCollection with different display scales to the .image function.

Consider the following test case:

func testPrimaryButtonStates() {
    // Given
    let frame = CGRect(x: 0, y: 0, width: 200, height: 48)
    let button = PrimaryButton(frame: frame)
    button.awakeFromNib()
    button.setTitle("PrimaryButton", for: .normal)

    // When
    button.testState = .enabled

    // Then
    assertSnapshot(matching: button, as: .image(traits: .init(displayScale: 2)), named: "enabled@2x")
    assertSnapshot(matching: button, as: .image(traits: .init(displayScale: 3)), named: "enabled@3x")
}

This test passes on Simulators with the same display scale (say iPhone 7 -> iPhone 8), but fails when the display scale changes (iPhone 7 -> iPhone X).
Since we run the tests on our CI on multiple devices, we can't guarantee the display scale to always be the same.

Here is a diff of the screenshots of one of the failing test cases:

screenshot 2018-12-07 at 13 19 34

Add support for running on CI when “build for testing” agent is different to testing agent

Firstly, really enjoying Snapshot tests. Already found a bug in our JSON parsing logic.

Problem

We compile our tests on one agent, then run the tests on various other agents in parallel. At compile time, #file is an absolute path to the file. At run time (on a different agent with a different project path), that path is not valid. Thus, all the tests try re-recording the snapshot and fail.

Suggestions

  1. Allow to look in Bundle resources for the snapshot file
  2. Allow a run-time file search to find the path of the snapshot

We’ve used resources for bundling JSON files in our test cases, so that is my preference here.
Either option would be “opt-in”, defaulting to your folder structure based approach.

Sent with GitHawk

[Question] Rearchitect to not require subclassing SnapshotTestCase?

We use Quick for testing and are therefore required to subclass QuickSpec, which precludes us from also subclassing SnapshotTestCase. Is it possible to architect this as a set of extensions to XCTestCase so that it can be used alongside other testing libraries that are less flexible?

I would be happy to put some work into this and put up a PR if you think it would be possible/valuable.

re-dupe the de-dupe of `dump`

Classes get de-duped in the output of dump:

class User {
  let id: Int
  let name: String
  init(id: Int, name: String) {
    self.id = id
    self.name = name
  }
}

let user = User(id: 2, name: "Blobby")

let users = [
  user,
  user,
  user,
]

dump(users)

//▿ 3 elements
//  ▿ __lldb_expr_45.User #0
//    - id: 2
//    - name: "Blobby"
//  ▿ __lldb_expr_45.User #0
//  ▿ __lldb_expr_45.User #0

would be nice to de-dupe that

Explore failing based off snapshot state

We could explore how performance tests and snapshot tests are alike: performance tests are just snapshot tests of how fast a test should take. In this way, .image(precision: 0.9) is similar in how it allows for a certain amount of variation before failing.

One idea, for example, is to define a strategy like .viewHierarchyDepth, which could snap merely an image of how complicated a view's hierarchy is. On repeat runs, if the view hierarchy gets deeper, it'll fail.

Activities are not available in SwiftPM projects (even on macOS) when using `swift test`

Hey folks, ran out of my timebox for this one, but I'm running on macOS, but using SwiftPM. This means one of the XCTContext.runActivity is running, but the closed-source XCTest isn't available. This means #if !os(Linux) isn't going to work when running via swift test .

Their check probably isn't available for us to use, if you want to test it out:

git clone https://github.com/danger/danger-swift.git
cd danger-swift
git checkout snapshots # or dc41e4e06eb2c994e0ece6fdc355386a212fc0fd
swift build
swift run komondor install
swift package generate-xcodeproj
open Danger.xcodeproj

swift test

Convert Data in snapshot to readable format

▿ httpBody: Optional
▿ some: 175 bytes
- count: 175
▿ pointer: 0x0000604000175610
- pointerValue: 105827995702800

should be actually: - httpBody: "comments=string&phone=string&type=string"

Simple ViewController UI snapshot testing workflow

I'm trying to do a simple UI snapshot testing using a ViewController with some UILabel text centered.

In the test file, I instantiate the ViewController and do an assertSnapshot. First time fails as expected and records the snapshot time in the filesystem as expected.
The second time I run it (it should pass) fails logging a mismatch between the sizes of the images.

Error: failed - Expected image@(375.0, 812.0) to match image@(250.0, 541.333333333333)

I have 2 issues/concerns here:

  • Images in the report are the same but somehow it compares different sizes so it reports a failing test.
  • The error log should report what filename is comparing to check manually in the filesystem. In this test, I only have 1 image but it can be trickier to find the proper image if I have several snapshots recorded.

`#if Xcode` compiler directive

I was integrating this into a project via Cocoapods and noticed that all the code between the #if Xcode directives wasn't being compiled. Where is this defined? I'm using Xcode, of course 😄

I worked around this by adding -D Xcode to OTHER_SWIFT_FLAGS for the library via a Podfile post_install hook.

Thank you for the awesome library

I just wanted to say thank you for designing this thing. It's fascinating, really.

I'm currently working on a PDF generation library for Linux, which is a perfect use case for SnapshotTesting, and it's amazing how I can compare the generated PDF files and get a nice diff in both string format (which is convenient when I want to be sure that the PDFs have exactly the same internal structure) and image format (to see if a change in code resulted in a visible regression).

Keep up the good work.

Support UIAppearance and visual effect views

iOSSnapshotTestCase supports UIAppearance and visual effect views using a window and drawViewHierarchyInRect:

https://github.com/uber/ios-snapshot-test-case/blob/de1c714952ca6b7f1a40670135297fd761ba43cf/FBSnapshotTestCase/Categories/UIImage%2BSnapshot.m#L39-L70

We typically write tests against framework, not app, targets, where I'm not sure this is even possible. We should investigate support, though, maybe at least by introspecting whether or not there's a window we can use.

Device simulator dependence on recursive description snapshots

I have found that recursive description snapshots produce different outcomes when tests run on different iOS simulators. Is that possible to make them repetitive no matter which device simulator I use?
I didn't find any solution on that yet. Maybe adding some configuration like on the image (i.e. .image(on: .iPhoneX)) will be useful here as well?

Add size configuration option to view snapshots

I'd be nice to configure the view size and it'd be nice to have some easily configured options, like device sizes and screen resolutions.

Responsive design mode has some nice options:

  • iPhone SE (320x568)
  • iPhone 8 (375x667)
  • iPhone 8 Plus (414x736) (3x size...can we fake it?)
  • iPad mini (7.9") / iPad (9.7") (1024x768)
  • iPad Pro (10.5") (1112x834)
  • iPad Pro (12.9") (1366x1024)
  • 800x600
  • 1366x768
  • 1920x1080

It'd also be nice to adapt this to have an "below the fold" option, which renders with an adaptive height and configurable fold rendering option.

Things to snapshot

  • NSAttributedString
  • SCNView
  • URL
  • URLRequest
  • UIView recursive description
  • signal
  • json
  • plist
  • html

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.