Flutter driver使用

Flutter 1.0问世后关注度越来越高,在github上已经5万颗星星了。前些日子看了下它的test tutorial已然支持了integration test -> flutter driver。起初看名字以为是flutter版的wedriver(再包装),研究了才发现另有一番天地。

Flutter的test分三个级别:

  • Unit test。单元测试,函数级别的逻辑验证(no-UI)。
  • Widget test。组件测试,组件的交互测试。
  • Integration test。集成测试,app内widgets的集成测试

说重点:

三种测试基础环境是一样的

Unit test调用的是’package:test’。原文:The package lets you run your tests in a local Dart VM with a headless version of the Flutter Engine, which supplies these libraries. Using this command you can run any test, whether it depends on Flutter libraries or not. Unit跑测试时default option是Dart VM, 也可以选择跑在某一个browser或者platform。具体介绍可以看测试文档

Widget test是UT的升级版,调用的’package:flutter_test’。Widget组件是flutter的布局核心(万物皆是widget),因此Widget test在flutter test中占有重要的地位。Widget test在VM(没错,widget test也跑在Dart VM里面)通过flutter engine render一个交互的可视化组件(UI)直接测试渲染和交互效果,准确性和效率都能得到保证。

Integration test是widget test的拓展版(后文分析),调用的是’package:flutter_driver’。其实整个flutter app就是一个大的widget,所以integration test主要聚焦widget聚合后的交互表现。flutter driver test build-in在项目里,运行时test app也会在Dart VM里面render,通过test script与其交互。

三种test对象都(可以)在VM里面运行,那么怎么跟VM互动呢?答案是Dart VM Service Protocol。它封装了JSON-RPC 2.0,来处理WebSocket request/response。

SERVICE EXTENSION模式

我们来看例子。Flutter test的目录结构大概长这样:

  • lib
    • main.dart
  • test_driver
    • tap.dart
    • tap_test.dart

'main.dart’是app的入口。Folder 'test_driver’用来专门用来装integration test。

‘tap.dart’:

import 'package:flutter_driver/driver_extension.dart';
import 'package:flutter_todo/main.dart' as app;

void main() {
  enableFlutterDriverExtension();
  app.main();
}

'tap.dart’是flutter driver的target file,它先enable了FlutterDriverExtension,再启动了app。enableFlutterDriverExtension()做了些什么呢:

注册了名为’ext.flutter.driver’的service extension。它调用了’flutter/foundation/binding.dart’中的registerServiceExtension(最终是调用dart/developer/extension中的service protocol extension handler),是_DriverBinding的一个function。

...
void initServiceExtensions() {
    super.initServiceExtensions();
    final FlutterDriverExtension extension = FlutterDriverExtension(_handler, _silenceErrors);
    registerServiceExtension(
      name: _extensionMethodName,
      callback: extension.call,
    );
  }
...

激活Flutter Driver VM service extension。

...
void enableFlutterDriverExtension({ DataHandler handler, bool silenceErrors = false }) {
  assert(WidgetsBinding.instance == null);
  _DriverBinding(handler, silenceErrors);
  assert(WidgetsBinding.instance is _DriverBinding);
}
...

接收来自FlutterDriver的command去调用相应的处理。代码太长不贴了,简单的说就是定义了 commandHandlers, commandDeserializers和_finders三个map映射来接收command,去调用它们的映射函数。handle这个过程的就是extension.call。所有在FlutterDriver中的方法最终都通过command映射到extension中对应的函数。

'tap_test.dart’是test script,下面是一个简单的flow。

‘tap_test.dart’:

import 'dart:async';

// Imports the Flutter Driver API
import 'package:flutter_driver/flutter_driver.dart';
import 'package:test/test.dart';

void main() {
  group('Tap test', () {
    FlutterDriver driver;

    setUpAll(() async {
      // Connects to the app
      driver = await FlutterDriver.connect();
    });

    tearDownAll(() async {
      if (driver != null) {
        // Closes the connection
        driver.close();
      }
    });

    test('measure', () async {
      // Record the performance timeline of things that happen inside the closure
      Timeline timeline = await driver.traceAction(() async {
        // Find the press button
        SerializableFinder pressButton = find.byTooltip('Increment');

        // Scroll down 5 times
        for (int i = 0; i < 5; i++) {
          // Tap for 300 millis
          await driver.tap(
              pressButton, timeout: Duration(milliseconds: 300));
        }
      });

      // The `timeline` object contains all the performance data recorded during
      // the session. It can be digested into a handful of useful
      // aggregate numbers, such as "average frame build time".
      TimelineSummary summary = TimelineSummary.summarize(timeline);

      // The following line saves the timeline summary to a JSON file.
      summary.writeSummaryToFile('tap_performance', pretty: true);

      // The following line saves the raw timeline data as JSON.
      summary.writeTimelineToFile('tap_performance', pretty: true);
    });
  });
}

初始化FlutterDriver后,driver做了几件事:

连接VM里面运行的test app。'connect()'中的关键部分如下,它去尝试连接VMServiceClient,拿到app的isolate(isolate可以认为是dart里面的多线程):

...
    final VMServiceClientConnection connection =
        await vmServiceConnectFunction(dartVmServiceUrl);
    final VMServiceClient client = connection.client;
    final VM vm = await client.getVM();
    final VMIsolateRef isolateRef = isolateNumber ==
        null ? vm.isolates.first :
               vm.isolates.firstWhere(
                   (VMIsolateRef isolate) => isolate.number == isolateNumber);
    _log.trace('Isolate found with number: ${isolateRef.number}');

    VMIsolate isolate = await isolateRef
        .loadRunnable()
        .timeout(isolateReadyTimeout, onTimeout: () {
      throw TimeoutException(
          'Timeout while waiting for the isolate to become runnable');
    });
...

在isolate中调用invokeExtension,将script里面的finder/interaction通过extension转化成command,来调用映射函数完成交互:

...
  Future<Map<String, dynamic>> _sendCommand(Command command) async {
    Map<String, dynamic> response;
    try {
      final Map<String, String> serialized = command.serialize();
      _logCommunication('>>> $serialized');
      response = await _appIsolate
          .invokeExtension(_flutterExtensionMethodName, serialized)
          .timeout(command.timeout + _rpcGraceTime(timeoutMultiplier));
      _logCommunication('<<< $response');
    } on TimeoutException catch (error, stackTrace) {
    ...
    } catch (error, stackTrace) {
    ...
    }
    ...
    return response['response'];
  }
...

一些细节

extension中查找元素都是通过flutter test的find实现的。它可以在render tree中搜索想要的widget。具体细节可以参考’flutter_test/lib/finder.dart’。

extension中的交互都是通过LiveWidgetController中的方式实现的。具体细节可以参考’flutter_test/lib/controller.dart’

方法forceGC()往VM发一条‘_collectAllGarbage’的命令强行GC。是不是remote interaction容易memory leak?

方法traceAction()收集performance timeline,提供了一个容易的方法track performance。结果长如下的样子:

{
  "average_frame_build_time_millis": 12.4331875,
  "90th_percentile_frame_build_time_millis": 18.181,
  "99th_percentile_frame_build_time_millis": 160.043,
  "worst_frame_build_time_millis": 160.043,
  "missed_frame_build_budget_count": 8,
  "average_frame_rasterizer_time_millis": 9.003393939393936,
  "90th_percentile_frame_rasterizer_time_millis": 9.284,
  "99th_percentile_frame_rasterizer_time_millis": 129.851,
  "worst_frame_rasterizer_time_millis": 129.851,
  "missed_frame_rasterizer_budget_count": 4,
  "frame_count": 32,
  "frame_build_times": [
    160043,
    6363,
    20967,
    ...
  ],
  "frame_rasterizer_times": [
    129851,
    15393,
    ...
  ]
}

有待更深入发掘…