Giter VIP home page Giter VIP logo

pactswift's People

Contributors

huwr avatar orj avatar surpher avatar tonyarnold avatar

Stargazers

 avatar

Watchers

 avatar

pactswift's Issues

Surface interaction mismatches to the test running the validation

๐Ÿ—ฃ Context

When setting up a Pact test and running it, lots of times the expected request does not match the request being triggered. When that happens, an informative message at the failing test should be surfaced up to the failing test.
At the moment it only fails with Server error 500 and debug area shows the expected and actual request which is difficult to filter through.

๐Ÿ’ฌ Narrative

When a request does not match the expectation
I want the context and reason for failing to be surfaced to the
So that it's easier for developer to understand why the test failed

๐Ÿ“ Notes

As a guide surpher/PactSwift surfaces the issue like so:

Screenshot 2023-05-11 at 12 31 26 pm

โœ… Acceptance Criteria

GIVEN a triggered request does not match the expected request defined in builder
WHEN PactSwift throws an error from libpact_ffi
THEN the line in the test is highlighted with XCTestFail and contains:

  • Reason for failure (eg: "Actual request does not match expected interaction")
  • Expected request (eg: POST /v1/endpointName)
  • Actual request (eg: POST /v1/endpointNmae)

Support matchers and example generators in requests

๐Ÿ—ฃ Context

Setting expectations for requests should be as strict as possible since we own the code and own the data to verify our system's behaviour - Choosing the right type of matching.

Typically, exact matching is most appropriate for Pact tests on the consumer 
side that are running at the unit test level. The same person is responsible for 
both the expectation and the actual request, so making sure that they match should 
be straightforward.

But sometimes when sending requests, specifically POST requests, sometimes we need to generate a random value and send to the provider. This can be an issue when we want to verify the key is being sent, but we can't guarantee the value to be sent.

๐Ÿ’ฌ Narrative

When setting up expectations for requests
I want to be able to use matchers
So that requests don't fail when generated values are being sent

๐Ÿ— Design

class SomePactTest: PactTestCase {
class SomePactTest: PactTestCase {

    func testAnInteraction() async throws {
        try builder
            .uponReceiving("A request to create a record")
            .given("A record does not exist")
            .withRequest(method: .POST, path: "/records") { request in
                // #start of feature request
                try request
                    .jsonBody(
                        .like([
                            "identifier": .uuid("b425456a-2450-43ab-86cf-1a8b9d3981c4", .lowerCaseHyphenated),
                            "anotherIdentifier": .randomUUID(), // using an example generator 
                            // https://github.com/pact-foundation/pact-specification/tree/version-3#introduce-example-generators
                            "type": .oneOf(["A", "B", "C"]),
                            "name": .like("some string"),
                            "value": .number(5)
                        ])
                    )   
                // #end         
            }
            .willRespond(with: 201) { resonse in
                try response.jsonBody(
                    .like([
                        "result": .like("ok")
                    ])
                )
            }
    }
}

โœ… Acceptance Criteria

GIVEN defining expectations for requests
WHEN matchers and/or example generators are used
THEN they are reflected in the Pact contract
and considered when verifying the interactions (running Pact test) (eg: request doesn't fail if expected uuid doesn't match the sent uuid)

๐Ÿšซ Out of Scope

Task: Find a way to reduce PactSwiftMockServer repo size

โ• Problem Statement

PactMockServer exposes the wrapper around libpact_ffi in a XCFramework package. But the libpact_ffi binaries build from Rust code (https://github.com/pact-foundation/pact-reference/tree/master/rust/pact_ffi) contains binaries for simulators and physical device. The problem is that all these binaries are static and include the rust runtime code for each of the platforms we're supporting (x86_64, aarm for each simulator and physical devices). These binaries are huge! They hover at just over 100MB of each. That also means a lot of bandwidth and long time to fetch PactSwift package. It also uses up unnecessary disk space on developers' machines.

Although it's nice to not worry about running Pact tests on a specific target, it doesn't really make sense to run them on a physical device since it doesn't make much sense trying to find where the Pact contract has been written to on the physical device and trying to extract it.

We can consider just failing terribly when a developer tries to run Pact tests on a physical device, or "gracefully" fail the test with a meaningful message.

surpher/PactSwift allows developers to run tests on a physical device, but skips writing the Pact contract onto iDevice's disk.

๐Ÿ’ก Or maybe look into having PactSwiftMockServer package only contain the source files and can we leverage SPM Plugins to fetch libpact_ffi binaries separately from repo? This could also let us drop dealing with rust altogether as we could potentially spm-plugin-execute fetching a binary from pact-foundation/pact-reference that's already been built before we build xcframework to vend from PactSwiftMockServer.
The thing is that each update to libpact_ffi "baked" into the repo just explodes the size that each project pulls in.
For example, pact-foundation/pact-reference is releasing these binaries and hosting them on GitHub, but we'll need to improve their script generating the static libs to use the right triples we would need (eg: aarch64-apple-ios-sim, x86_64-apple-ios and aarch64-apple-darwin, x86_64-apple-darwin).

Another option to reduce the libpact_ffi size for macOS targets would be to share a dynamic lib? The released FFI libs pact-reference/rust offers are around 7MB in size.
But moving to a dynamic lib for macOS would mean developers would also need to install rust on their machines? This could prove as a big barrier to adoption of PactSwift.

Reach out to me (@surpher) and we can talk about all of the approaches I've thought up. There's been quite a few, and a few approaches I've already tried I am really not proud of!

๐Ÿ’ฌ Task Description

  1. Remove support to run on physical devices.
  2. Look for ways to not have to commit XCFrameworks and lib.as into repo

๐Ÿ‘ฉโ€๐Ÿ”ง Technical Design Notes

๐Ÿค Relationships

  • Other Related Issues:

Task: Unit tests

โ• Problem Statement

This fork has been spun up for specific needs of a project IBA is involved in. Therefore some of the code is not tested properly.

๐Ÿ’ฌ Task Description

๐Ÿ‘ฉโ€๐Ÿ”ง Technical Design Notes

๐Ÿค Relationships

  • Other Related Issues: #xxx, #yyy

Task: POC using plugins

โ• Problem Statement

Pact supports plugins https://github.com/pact-foundation/pact-plugins.

๐Ÿ’ฌ Task Description

Look into how to make use of it

๐Ÿ‘ฉโ€๐Ÿ”ง Technical Design Notes

From what's been seen in other Pact implementations it could look something like:

try builder
   .uponReceiving(...)
   .given(...)
   .usingPlugin(plugin_name) // <- This little diddy
   .withRequest(...) { ... }
   ...

๐Ÿค Relationships

  • Other Related Issues: #xxx, #yyy

Task: Clean up tests and ObjC/Linux specific code

โ• Problem Statement

surpher/PactSwift exposes Objective-C interface and supports using this framework on Linux platforms.

๐Ÿ’ฌ Task Description

Go through the code and remove any ObjC and Linux support. Most of it has already been done though.

๐Ÿ‘ฉโ€๐Ÿ”ง Technical Design Notes

/

๐Ÿค Relationships

  • Other Related Issues:

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.