Giter VIP home page Giter VIP logo

fast_flutter_driver's Introduction

Fast Flutter Driver

Build codecov

Plugin Pub
Command Line Tool pub package
UI Helper pub package

Deprecation Warning

Flutter Driver tests are being deprecated in favor of new Integration Tests. I will continue to support this package as long it's feasible but it is recommended to start using Integration Tests.

You can read more Pros and Cons here.

Toolkit for running rapidly flutter driver tests on desktop/mobile.

This package simplifies the creation of Flutter Driver tests that can be run in bulk without restarting or rebuilding the application. An example of how the toolkit works can be found in this article.

The reasoning for Desktop tests

The desktop builds are almost identical to Android/iOS builds in regard to the UI. Running tests on a simulator/emulator are painful. This can be even worse on your CI.

Your CI is usually a dockerized Linux machine - running Linux flutter driver tests against Linux Flutter application is both fast and reliable. This tool allows you to run your flutter driver tests much faster and more reliably than a build for a simulator/emulator.

The application flows that require Android/iOS plugins should still be run on a Simulator/Emulator or a real device.

Running also on Android/iOS

While running tests against Desktop will find the majority of bugs, it is recommended to run sometimes against actual devices. There are bugs in Dart (like implementing your generics in an invalid way) that can cause a native crash on a mobile device and work properly on the desktop.It's prudent to run fastdriver tests before every release on a mobile device.

Example

You can build and execute the example that works on any desktop system and Docker as well.

Usage

  • Add dev_dependency to your pubspec.yaml
dev_dependencies:
  fast_flutter_driver: ^2.0.0
  • Create configuration class test_driver/generic/test_configuration.dart
import 'package:fast_flutter_driver/tool.dart';
import 'package:meta/meta.dart';

class TestConfiguration implements BaseConfiguration {
  const TestConfiguration({
    required this.resolution,
    this.platform,
  });

  factory TestConfiguration.fromJson(Map<String, dynamic> json) {
    return TestConfiguration(
      resolution: Resolution.fromJson(json['resolution']),
      platform: TestPlatformEx.fromString(json['platform']),
    );
  }
  @override
  final TestPlatform? platform;
  @override
  final Resolution resolution;
  
  @override
  Map<String, dynamic> toJson() {
    final p = platform;
    return <String, dynamic>{
      'resolution': resolution,
      if (p != null) 'platform': p.asString(),
    };
  }
}
  • Create dart file test_driver/generic/generic.dart with content and MyApplication as your main (root) application widget.
import 'dart:convert';

import 'package:fast_flutter_driver/driver.dart';
import 'package:flutter/foundation.dart';
import 'package:flutter/scheduler.dart';
import 'package:flutter/widgets.dart';
import 'package:flutter_driver/driver_extension.dart';

import 'test_configuration.dart';

void main() {
  timeDilation = 0.1;
  enableFlutterDriverExtension(
    handler: (playload) async {
      await configureTest(
        TestConfiguration.fromJson(json.decode(playload ?? '{}')),
      );
      return '';
    },
  );

  runApp(
    RestartWidget<TestConfiguration>(
      builder: (_, config) => MyApplication(),
    ),
  );
}
  • Create a test, eg test_driver/example_test.dart
import 'dart:convert';

import 'package:fast_flutter_driver/tool.dart';
import 'package:flutter_driver/flutter_driver.dart';
import 'package:test/test.dart';

import 'generic/test_configuration.dart';

void main(List<String> args) {
  late FlutterDriver driver;
  final properties = TestProperties(args);

  setUpAll(() async {
    driver = await FlutterDriver.connect(dartVmServiceUrl: properties.vmUrl);
  });

  tearDownAll(() async {
    await driver.close();
  });

  Future<void> restart() {
    return driver.requestData(
      json.encode(
        TestConfiguration(
          resolution: properties.resolution,
          platform: properties.platform,
        ),
      ),
    );
  }

  test('main application', () async {
    await restart();

    await driver.waitFor(find.byType('MyApplication'));
  });
}
pub global activate fast_flutter_driver_tool
  • Run:
fastdriver --dart-args "--no-sound-null-safety" --flutter-args "--no-sound-null-safety"

All done!

What's next

This was the simplest setup of tests, next you would like to pass different application configuration so every time you run tests, they could be run with a different configuration (eg different page). To do that you need to add more properties to TestConfiguration.

A full example of how to do that can be found in the example folder.

Null Safety

The package supports building project that are migrated to null safety but until flutter driver is migrated to null safety, we need to pass the above flags to the command line tool.

fast_flutter_driver's People

Contributors

akaiser avatar samandmoore avatar tomaszpolanski avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

fast_flutter_driver's Issues

Should not be used in libraries (packages)

This package depends on window_utils package but uses a github version as the latest one has a certain issue with Linux.
Because git link is used instead of a pub.dev package, this package cannot be included in other libraries.

A solution would be to include a modified version of window_utils inside (Rody - the creator - already have his thumbs up).

flutter_driver from sdk is incompatible with fast_flutter_driver

Running with SDK version 2.10.0 as well as 2.12.0, I get this error when keeping flutter_driver: sdk: flutter in my pubspec. the conflict ist on the args package, flutter_driver seems to depend on 1.6.0 even with the 2.12.0 SDK version. I haven't found any open or closed issues on this, which SDK version is everyone else using?

Crash when running `dev` channel (1.24.0-3.0.pre)

Hi there,

was trying out your library but got stuck on the following error:

is the library supposed to work with this Flutter version (is there a way to pass the non-nullable experiment flag?) or do I need a specific version?

 dart test_driver/example_test.dart -u http://127.0.0.1:63182/wysSQ6jph-I=/ -r 400x700 -l en
../../../development/flutter-stable/packages/flutter_driver/lib/flutter_driver.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.8.
// Copyright 2014 The Flutter Authors. All rights reserved.
^
../../../development/flutter-stable/packages/flutter_test/lib/flutter_test.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.8.
// Copyright 2014 The Flutter Authors. All rights reserved.
^
../../../development/flutter-stable/packages/flutter_driver/lib/src/common/deserialization_factory.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.8.
// Copyright 2014 The Flutter Authors. All rights reserved.
^
../../../development/flutter-stable/packages/flutter_driver/lib/src/common/diagnostics_tree.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.8.
// Copyright 2014 The Flutter Authors. All rights reserved.
^
../../../development/flutter-stable/packages/flutter_driver/lib/src/common/diagnostics_tree.dart:31:13: Error: This requires the 'non-nullable' experiment to be enabled.
Try enabling this experiment by adding it to the command line when compiling and running.
    Duration? timeout,
            ^
../../../development/flutter-stable/packages/flutter_driver/lib/src/common/enum_util.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.8.
// Copyright 2014 The Flutter Authors. All rights reserved.
^
../../../development/flutter-stable/packages/flutter_driver/lib/src/common/error.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.8.
// Copyright 2014 The Flutter Authors. All rights reserved.
^
../../../development/flutter-stable/packages/flutter_driver/lib/src/common/find.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.8.
// Copyright 2014 The Flutter Authors. All rights reserved.
^
../../../development/flutter-stable/packages/flutter_driver/lib/src/common/find.dart:26:43: Error: This requires the 'non-nullable' experiment to be enabled.
Try enabling this experiment by adding it to the command line when compiling and running.
  CommandWithTarget(this.finder, {Duration? timeout}) : super(timeout: timeout) {
                                          ^
../../../development/flutter-stable/packages/flutter_driver/lib/src/common/find.dart:57:47: Error: This requires the 'non-nullable' experiment to be enabled.
Try enabling this experiment by adding it to the command line when compiling and running.
  WaitFor(SerializableFinder finder, {Duration? timeout})

Flutter 2.5.0 breaks fast_flutter_driver_tool

When attempting to use the fastdriver tool on windows, I see the following error:

Unhandled exception:
Bad state: Future already completed

This is occurring because the latest version of flutter prints "is available at" multiple times when starting the app in debug mode. The tool can be fixed by modifying the nativeMatch variable in the _buildAndRun method in testing.dart:

final nativeMatch = RegExp(
              'An Observatory debugger and profiler on .* is available at: (http://.*/)')
          .firstMatch(line);

unable to get tests to work

Hi Tomas, I have been working many hours to get your package to work and I seem to be at a standstill.

First - I updated pubspec.yaml but was not sure from your documentation and example whether I update either dependencies with fast_flutter-driver_tool ^1.5.1 or dev_dependences with fast_flutter_driver ^1.1.0

I tried one then both and get the same result.

My app works fine running macos with no errors.

I setup three tests to run in test_driver and the test starts the generic_test.dart file...then starts my app...but then stops and provides a number of errors that don't help me in debugging where to resolve:

Here is print out of the run:

brians-mbp:master brianremington$ fastdriver
Can't load Kernel binary: Invalid kernel binary format version.
Starting tests
Testing test_driver/generic/generic_test.dart
Building application for macos... 45.3s
Syncing files...
Syncing files... 0.0s
$ dart test_driver/generic/generic_test.dart -u http://127.0.0.1:65153/7HI__byYJQI=/ -r 400x700 -l en
../../flutter/.pub-cache/hosted/pub.dartlang.org/test-1.16.0-nullsafety.5/lib/test.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.9.
// Copyright (c) 2013, the Dart project authors. Please see the AUTHORS file
^
../../flutter/packages/flutter/lib/material.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.9.
// Copyright 2014 The Flutter Authors. All rights reserved.
^
../../flutter/.pub-cache/hosted/pub.dartlang.org/test_core-0.3.12-nullsafety.5/lib/test_core.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.9.
// Copyright (c) 2013, the Dart project authors. Please see the AUTHORS file
^
../../flutter/packages/flutter/lib/foundation.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.9.
// Copyright 2014 The Flutter Authors. All rights reserved.
^
../../flutter/packages/flutter/lib/cupertino.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.9.
// Copyright 2014 The Flutter Authors. All rights reserved.
^
../../flutter/packages/flutter/lib/services.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.9.
// Copyright 2014 The Flutter Authors. All rights reserved.
^
../../flutter/.pub-cache/hosted/pub.dartlang.org/meta-1.3.0-nullsafety.3/lib/meta.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.9.
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
^
../../flutter/.pub-cache/hosted/pub.dartlang.org/file-6.0.0-nullsafety.2/lib/file.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.9.
// Copyright (c) 2017, the Dart project authors. Please see the AUTHORS file
^
../../flutter/.pub-cache/hosted/pub.dartlang.org/file-6.0.0-nullsafety.2/lib/local.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.9.
// Copyright (c) 2017, the Dart project authors. Please see the AUTHORS file
^
../../flutter/.pub-cache/hosted/pub.dartlang.org/file-6.0.0-nullsafety.2/lib/memory.dart:1:1: Error: The specified language version is too high. The highest supported language version is 2.9.
// Copyright (c) 2017, the Dart project authors. Please see the AUTHORS file
^
Unhandled exception:
ShellException(dart test_driver/generic/generic_test.dart -u http://127.0.0.1:65153/7HI__byYJQI=/ -r 400x700 -l en, exitCode 254, workingDirectory: /Users/brianremington/Downloads/master)
#0 Shell.runExecutableArguments (package:process_run/shell.dart:274:9)

#1 Shell.run (package:process_run/shell.dart:241:33)
#2 _CommandLine.run (package:fast_flutter_driver_tool/src/preparing_tests/command_line/command_line_impl.dart:44:18)
#3 run (package:fast_flutter_driver_tool/src/preparing_tests/command_line/command_line_impl.dart:24:5)
#4 TestExecutor._runTests (package:fast_flutter_driver_tool/src/preparing_tests/testing.dart:182:16)
#5 TestExecutor.test (package:fast_flutter_driver_tool/src/preparing_tests/testing.dart:110:15)

#6 run. (file:///Users/brianremington/.pub-cache/hosted/pub.dartlang.org/fast_flutter_driver_tool-1.5.1/bin/main.dart:102:41)
#7 setUp (package:fast_flutter_driver_tool/src/preparing_tests/testing.dart:31:16)
#8 run (file:///Users/brianremington/.pub-cache/hosted/pub.dartlang.org/fast_flutter_driver_tool-1.5.1/bin/main.dart:100:11)

#9 main (file:///Users/brianremington/.pub-cache/hosted/pub.dartlang.org/fast_flutter_driver_tool-1.5.1/bin/main.dart:23:9)
#10 _startIsolate. (dart:isolate-patch/isolate_patch.dart:299:32)
#11 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)

Here is one of the _test files:

import 'dart:convert';
import 'package:fast_flutter_driver/tool.dart';
import 'package:flutter_driver/flutter_driver.dart';
import 'package:test/test.dart';
import 'package:seed/main.dart';
import '../test_driver/generic/test_configuration.dart';

void main(List args) {
FlutterDriver driver;
final properties = TestProperties(args);

setUpAll(() async {
driver = await FlutterDriver.connect(
dartVmServiceUrl: properties.vmUrl,
);
});

tearDownAll(() async {
await driver?.close();
});

Future restart() {
db.resetApp();
return driver.requestData(
json.encode(
TestConfiguration(
resolution: properties.resolution,
platform: properties.platform,

    ),
  ),
);

}

group('all test scenarios', () {
SerializableFinder nextIntroScreen = find.byValueKey('next');
SerializableFinder doneIntroScreen = find.byValueKey('done');
SerializableFinder introScreen = find.byValueKey('introStart');
SerializableFinder assetGotIt = find.byValueKey('assetGotIt');
SerializableFinder assetChip = find.byValueKey('assetChip');
SerializableFinder housing = find.byValueKey('cardHousing');
SerializableFinder savings = find.byValueKey('cardSavings');
SerializableFinder k4011 = find.byValueKey('cardEmployer plan 1');
SerializableFinder salary1Card = find.byValueKey('cardSalary 1');
SerializableFinder salary1FTKeyboard = find.byValueKey('sal1FTKeyboard');
SerializableFinder salary1Field = find.byValueKey('salary1Field');
SerializableFinder firstHouseListTile = find.byValueKey('firstHouseListTile');
SerializableFinder key7 = find.byValueKey('key7');
SerializableFinder key5 = find.byValueKey('key5');
SerializableFinder key0 = find.byValueKey('key0');
SerializableFinder doneButtonKeyboard = find.byValueKey('doneButtonKeyboard');
SerializableFinder doneButtonSalary = find.byValueKey('doneButtonSalary');
SerializableFinder doneButtonSavings = find.byValueKey('doneButtonSavings');
SerializableFinder doneButton401k = find.byValueKey('doneButton401k');
SerializableFinder doneButtonLoanPop = find.byValueKey('doneButtonLoanPop');
SerializableFinder doneButtonAsset = find.byValueKey('doneButtonAsset');
SerializableFinder assetCardBackButton = find.byValueKey('assetCardBackButton');
SerializableFinder currentHomeValueKeyboard = find.byValueKey('currentHomeValueKeyboard');
SerializableFinder currentHomeLoanKeyboard = find.byValueKey('currentHomeLoanKeyboard');
SerializableFinder savingsChip = find.byValueKey('savingsChip');
SerializableFinder k401Chip = find.byValueKey('k401Chip');
SerializableFinder savingsKeyboard = find.byValueKey('savingsKeyboard');
SerializableFinder k401Keyboard = find.byValueKey('k401Keyboard');

test('USAge30Single', () async {
  await driver.waitFor(introScreen);
  await driver.tap(nextIntroScreen, timeout: Duration(seconds: 1));
  await driver.tap(nextIntroScreen, timeout: Duration(seconds: 1));
  await driver.tap(nextIntroScreen, timeout: Duration(seconds: 1));
  await driver.tap(nextIntroScreen, timeout: Duration(seconds: 1));
  await driver.tap(nextIntroScreen, timeout: Duration(seconds: 1));
  await driver.tap(salary1Card, timeout: Duration(seconds: 1));
  await driver.tap(salary1FTKeyboard, timeout: Duration(seconds: 1));
  await driver.tap(key7);
  await driver.tap(key5);
  await driver.tap(key0);
  await driver.tap(key0);
  await driver.tap(key0);
  await driver.tap(doneButtonKeyboard, timeout: Duration(seconds: 2));

  // test that salary works
  await driver.waitFor(find.text("\$75,000"));
  expect(await driver.getText(salary1Field), "\$75,000");
  await driver.tap(doneButtonSalary);
  //test for social security starting at retire age
  await expectLater(db.getSalary1RetireValue(), completion(38438));
  //test for taxes complied correctly
  //taxes based on 2021 US tax tables for single living in CA
  await expectLater(db.getTaxValue(0), completion(-18423));
  //test for salary stopping at retire age for those under age 65
  await expectLater(db.getSalary1RetireValue(), completion(-100));
  //test for initial expenses estimated
  await expectLater(db.getValueByYearAndName("Food",0), completion(-7361));
  await driver.tap(doneIntroScreen, timeout: Duration(seconds: 1));
});

test('add savings', () async {
  await driver.tap(savings, timeout: Duration(seconds: 1));
  await driver.tap(savingsChip, timeout: Duration(seconds: 1));
  await driver.tap(savingsKeyboard, timeout: Duration(seconds: 1));
  await driver.tap(key5);
  await driver.tap(key0);
  await driver.tap(key0);
  await driver.tap(key0);
  await driver.tap(key0);
  await driver.tap(doneButtonKeyboard, timeout: Duration(seconds: 1));
  await driver.tap(doneButtonSavings);
  //test for savings at end of model
  await expectLater(db.getValueByYearAndName("Savings",39), completion(108214));
});

test('add 401k', () async {
  await driver.tap(k4011, timeout: Duration(seconds: 1));
  await driver.tap(k401Chip, timeout: Duration(seconds: 1));
  await driver.tap(k401Keyboard, timeout: Duration(seconds: 1));
  await driver.tap(key5);
  await driver.tap(key0);
  await driver.tap(key0);
  await driver.tap(key0);
  await driver.tap(key0);
  await driver.tap(doneButtonKeyboard, timeout: Duration(seconds: 1));
  await driver.tap(doneButton401k);
  //test for retirement account at end of model
  await expectLater(db.getValueByYearAndName("Employer plan 1",39), completion(349959));

});

test('add existing house', () async {
  await driver.tap(assetChip, timeout: Duration(seconds: 1));
  await driver.tap(assetGotIt, timeout: Duration(seconds: 1));
  await driver.tap(housing, timeout: Duration(seconds: 1));
  await driver.tap(firstHouseListTile, timeout: Duration(seconds: 1));
  await driver.tap(currentHomeValueKeyboard, timeout: Duration(seconds: 1));
  await driver.tap(key5);
  await driver.tap(key7);
  await driver.tap(key0);
  await driver.tap(key0);
  await driver.tap(key0);
  await driver.tap(key0);
  await driver.tap(doneButtonKeyboard, timeout: Duration(seconds: 1));
  await driver.tap(currentHomeLoanKeyboard, timeout: Duration(seconds: 1));
  await driver.tap(key5);
  await driver.tap(key5);
  await driver.tap(key0);
  await driver.tap(key0);
  await driver.tap(key0);
  await driver.tap(key0);
  await driver.tap(doneButtonKeyboard, timeout: Duration(seconds: 1));
  await driver.tap(doneButtonLoanPop);
  await driver.tap(doneButtonAsset);
  await driver.tap(assetCardBackButton);
  // check FV of home with appreciation
  await expectLater(db.getValueByYearAndName("Housing",39), completion(1233904));
  //check mortgage is being paid down..last year of payments
  await expectLater(db.getValueByYearAndName("Mortgage",29), completion(-34482));
  //check mortgage is paid off
  await expectLater(db.getValueByYearAndName("Mortgage",39), completion(0));

});

group('restart', () {
  test('restart', () async {
    await restart();
  });

});

});

}

Use fast_flutter_driver alongside flutter driver

I'd like to be able to run fast_driver and flutter driver on the same project. However, it seems as though the auto-generated fast_driver code clashes with flutter driver code. I was hoping that I could make a folder named fastdriver in the root of my project and leave existing tests in the test_driver directory. However, the two testers seem to clash in multiple ways.

Is this supported?

Ordering of tests

Is there any way to order the tests run in generic_test.dart short of hackily naming them with number prefixes or similar?

I tried hardcoding a generic_test.dart file, but it's overwritten when calling fastdriver.

The specific use case is ensuring login tests come first, and logout tests last, to prevent having to login in each test suite, and then I can just restart the app in the intermediary suites and maintain logged in state.

[Q] Is it possible to use this with an Async app

Looking at all the docs and testing it out I cannot see a way to get around required a Widget to be passed into the generic setup, however, I make sue of an Async app, which has quite a bit of bootstrap logic that needs to run before loading the main apps widgets.

WIthout calling app.main(), the app cannot run in the driver side and passing that into the setup does not work.

Thanks.

Flutter Web "fastdriver" command not working

Hi!

Im building a web application and would like to use fast_flutter_driver for testing.
I wen over the tutorial, added generic classes and fastdriver dependency.

Upon calling fastdriver -d chrome im getting puch of errors coming from .pub-cache/hosted/pub.dartlang.org/ffi and win directories stating that virtually nothing in there is found or isnt a type.

Blinking GIF in README

Hey! I just randomly stumbled upon your repository. I have a suggestion to hide the blinking GIF from the README behind a link with a clear warning that it contains blinking screens or showing this GIF with a slightly different example, maybe with different emojis on each screen or much less hue variance of the background.

Cheers!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.