Technical dry goods | Flutter online programming practice summary

1.Flutter architecture

Flutter's architecture is mainly divided into three layers: Framework, Engine, and Embedder.

1. The Framework is implemented using dart, including Material Design style Widget, Cupertino (for iOS) style Widgets, text/picture/button and other basic Widgets, rendering, animation, gestures, etc. The core code of this part is: flutter package under the flutter warehouse, and packages such as io, async, ui under the sky_engine warehouse (dart:ui library provides the interface between the Flutter framework and the engine).

2. Engine is implemented in C++, mainly including Skia, Dart and Text. Skia is an open source two-dimensional graphics library that provides a common API suitable for a variety of software and hardware platforms.

3. Embedder is an embedding layer, that is, embedding Flutter on various platforms. The main work done here includes rendering Surface settings, thread settings, and plug-ins. It can be seen from this that the platform-related layer of Flutter is very low, the platform (such as iOS) only provides a canvas, and all the remaining rendering-related logic is inside Flutter, which makes it have a good cross-end consistency.

2. Flutter view drawing

For developers, the framework is the most used. I will start with the entry function of Flutter and go down step by step to analyze the principle of drawing the Flutter view.

In the Flutter application, the simplest implementation of the main() function is as follows

// 参数app是一个widget，是Flutter应用启动后要展示的第一个Widget。
void runApp(Widget app) {
  WidgetsFlutterBinding.ensureInitialized()
    ..scheduleAttachRootWidget(app)
    ..scheduleWarmUpFrame();
}

1.WidgetsFlutterBinding

WidgetsFlutterBinding inherits from BindingBase and mixes into many Bindings. Looking at the source code of these Bindings, you can find that these Bindings basically monitor and process some events of the Window object (including some information of the current device and system and some callbacks of the Flutter Engine), and then These events are packaged, abstracted and distributed according to the Framework model.

WidgetsFlutterBinding is the "glue" that connects the Flutter engine and the upper Framework.

1. GestureBinding: Provides the window.onPointerDataPacket callback, binds the Framework gesture subsystem, and is the binding entry for the Framework event model and the underlying events.
2. ServicesBinding: Provides window.onPlatformMessage callback, used to bind platform message channel (message channel), mainly dealing with native and Flutter communication.
3. SchedulerBinding: Provides window.onBeginFrame and window.onDrawFrame callbacks, monitors refresh events, and binds the Framework drawing scheduling subsystem.
4. PaintingBinding: Binding drawing library, mainly used to handle picture caching.
5. SemanticsBinding: The bridge between the semantic layer and the Flutter engine, mainly the underlying support for auxiliary functions.
6. RendererBinding: Provides callbacks such as window.onMetricsChanged and window.onTextScaleFactorChanged. It is a bridge between the render tree and the Flutter engine.
7. WidgetsBinding: Provides callbacks such as window.onLocaleChanged and onBuildScheduled. It is the bridge between the Flutter widget layer and the engine.

WidgetsFlutterBinding.ensureInitialized() is responsible for initializing a global singleton of WidgetsBinding, the code is as follows

class WidgetsFlutterBinding extends BindingBase with GestureBinding, ServicesBinding, SchedulerBinding, PaintingBinding, SemanticsBinding, RendererBinding, WidgetsBinding {
  static WidgetsBinding ensureInitialized() {
    if (WidgetsBinding.instance == null)
      WidgetsFlutterBinding();
    return WidgetsBinding.instance;
  }
}

Seeing this WidgetsFlutterBinding mixed with many Bindings, let's first look at the parent class BindingBase:

abstract class BindingBase {
   ...
  ui.SingletonFlutterWindow get window => ui.window;//获取window实例
  @protected
  @mustCallSuper
  void initInstances() {
    assert(!_debugInitialized);
    assert(() {
      _debugInitialized = true;
      return true;
    }());
  }
}

See the code Window get window => ui.window to link the interface of the host operating system, that is, the interface of the Flutter framework to link the host operating system. There is a Window instance in the system, which can be obtained from the window property, look at the source code:

// window的类型是一个FlutterView，FlutterView里面有一个PlatformDispatcher属性
ui.SingletonFlutterWindow get window => ui.window;
// 初始化时把PlatformDispatcher.instance传入，完成初始化
ui.window = SingletonFlutterWindow._(0, PlatformDispatcher.instance);
// SingletonFlutterWindow的类结构
class SingletonFlutterWindow extends FlutterWindow {
  ...
  // 实际上是给platformDispatcher.onBeginFrame赋值
  FrameCallback? get onBeginFrame => platformDispatcher.onBeginFrame;
  set onBeginFrame(FrameCallback? callback) {
    platformDispatcher.onBeginFrame = callback;
  }
  
  VoidCallback? get onDrawFrame => platformDispatcher.onDrawFrame;
  set onDrawFrame(VoidCallback? callback) {
    platformDispatcher.onDrawFrame = callback;
  }
  
  // window.scheduleFrame实际上是调用platformDispatcher.scheduleFrame()
  void scheduleFrame() => platformDispatcher.scheduleFrame();
  ...
}
class FlutterWindow extends FlutterView {
  FlutterWindow._(this._windowId, this.platformDispatcher);
  final Object _windowId;
  // PD
  @override
  final PlatformDispatcher platformDispatcher;
  @override
  ViewConfiguration get viewConfiguration {
    return platformDispatcher._viewConfigurations[_windowId]!;
  }
}

2.scheduleAttachRootWidget

scheduleAttachRootWidget will then call the attachRootWidget method of WidgetsBinding, which is responsible for adding the root Widget to the RenderView. The code is as follows:

 void attachRootWidget(Widget rootWidget) {
    final bool isBootstrapFrame = renderViewElement == null;
    _readyToProduceFrames = true;
    _renderViewElement = RenderObjectToWidgetAdapter<RenderBox>(
      container: renderView,
      debugShortDescription: '[root]',
      child: rootWidget,
    ).attachToRenderTree(buildOwner!, renderViewElement as RenderObjectToWidgetElement<RenderBox>?);
    if (isBootstrapFrame) {
      SchedulerBinding.instance!.ensureVisualUpdate();
    }
  }

The renderView variable is a RenderObject, which is the root of the render tree. The renderViewElement variable is the Element object corresponding to the renderView. It can be seen that this method mainly completes the entire association process from the root widget to the root RenderObject and then to the root Element.

RenderView get renderView => _pipelineOwner.rootNode! as RenderView;

RenderView is the PipelineOwner.rootNode obtained in RendererBinding. PipelineOwner plays an important role in Rendering Pipeline:

As the UI changes, the "Dirty Render Objects" are continuously collected and then the Rendering Pipeline is driven to refresh the UI.

Simply put, PipelineOwner is a bridge between "RenderObject Tree" and "RendererBinding".

Finally, attachRootWidget is called, and the execution will call the attachToRenderTree method of RenderObjectToWidgetAdapter. This method is responsible for creating the root element, namely RenderObjectToWidgetElement, and associating the element with the widget, that is, creating the element tree corresponding to the widget tree. If the element has already been created, set the associated widget in the root element as a new one. It can be seen that the element will only be created once and will be reused later. BuildOwner is the management class of the widget framework, which tracks which widgets need to be rebuilt. code show as below

RenderObjectToWidgetElement<T> attachToRenderTree(BuildOwner owner, [RenderObjectToWidgetElement<T> element]) {
  if (element == null) {
    owner.lockState(() {
      element = createElement();
      assert(element != null);
      element.assignOwner(owner);
    });
    owner.buildScope(element, () {
      element.mount(null, null);
    });
  } else {
    element._newWidget = this;
    element.markNeedsBuild();
  }
  return element;
}
 

3.scheduleWarmUpFrame

In the implementation of runApp, when attachRootWidget is called, the last line will call the scheduleWarmUpFrame() method of the WidgetsFlutterBinding instance. The implementation of this method is in the SchedulerBinding. It will draw once immediately after it is called (instead of waiting for the "vsync" signal), Before the end of the drawing, this method will lock the event distribution, which means that Flutter will not respond to various events before the end of the drawing, which can ensure that no new redraws will be triggered during the drawing process.

The following is a partial implementation of the scheduleWarmUpFrame() method (irrelevant code is omitted):

void scheduleWarmUpFrame() {
  ...
  Timer.run(() {
    handleBeginFrame(null); 
  });
  Timer.run(() {
    handleDrawFrame();  
    resetEpoch();
  });
  // 锁定事件
  lockEvents(() async {
    await endOfFrame;
    Timeline.finishSync();
  });
 ...
}

This method mainly calls handleBeginFrame() and handleDrawFrame() two methods

Looking at the source code of the handleBeginFrame() and handleDrawFrame() methods, you can find that the former mainly executes the transientCallbacks queue, while the latter executes the persistentCallbacks and postFrameCallbacks queues.

1. transientCallbacks：用于存放一些临时回调，一般存放动画回调。

可以通过SchedulerBinding.instance.scheduleFrameCallback 添加回调。

2. persistentCallbacks：用于存放一些持久的回调，不能在此类回调中再请求新的绘制帧，持久回调一经注册则不能移除。

SchedulerBinding.instance.addPersitentFrameCallback()，这个回调中处理了布局与绘制工作。

3. postFrameCallbacks：在Frame结束时只会被调用一次，调用后会被系统移除，可由 SchedulerBinding.instance.addPostFrameCallback() 注册。

注意，不要在此类回调中再触发新的Frame，这可以会导致循环

The real rendering and drawing logic is implemented in RendererBinding. Looking at its source code, it is found that there are the following codes in its initInstances() method:

void initInstances() {
  ... // 省略无关代码
  addPersistentFrameCallback(_handlePersistentFrameCallback);
}
void _handlePersistentFrameCallback(Duration timeStamp) {
  drawFrame();
}
void drawFrame() {
  assert(renderView != null);
  pipelineOwner.flushLayout(); // 布局
  pipelineOwner.flushCompositingBits(); //重绘之前的预处理操作，检查RenderObject是否需要重绘
  pipelineOwner.flushPaint(); // 重绘
  renderView.compositeFrame(); // 将需要绘制的比特数据发给GPU
  pipelineOwner.flushSemantics(); // this also sends the semantics to the OS.
}

It should be noted that: Because RendererBinding is just a mixin, and with it is WidgetsBinding, so you need to see whether this method is rewritten in WidgetsBinding, and check the source code of the drawFrame() method of WidgetsBinding:

@override
void drawFrame() {
 ...//省略无关代码
  try {
    if (renderViewElement != null)
      buildOwner.buildScope(renderViewElement); 
    super.drawFrame(); //调用RendererBinding的drawFrame()方法
    buildOwner.finalizeTree();
  } 
}

Before calling the RendererBinding.drawFrame() method, buildOwner.buildScope() will be called (not the first draw), this method will rebuild() the elements marked as "dirty"
Let's look at WidgetsBinding again, create a BuildOwner object in the initInstances() method, and then execute buildOwner!.onBuildScheduled = _handleBuildScheduled; , where _handleBuildScheduled is assigned to the onBuildScheduled property of buildOwnder.

The BuildOwner object, which is responsible for tracking which widgets need to be rebuilt, and processing other tasks applied to the widgets tree, maintains a _dirtyElements list internally to save the elements marked as "dirty".

When each element is created, its BuildOwner is determined. A page has only one buildOwner object, which is responsible for managing all the elements of the page.

// WidgetsBinding
void initInstances() {
  ...
  buildOwner!.onBuildScheduled = _handleBuildScheduled;
  ...
  }());
}

当调用buildOwner.onBuildScheduled()时，便会走下面的流程。

// WidgetsBinding类
void _handleBuildScheduled() {
  ensureVisualUpdate();
}
// SchedulerBinding类
void ensureVisualUpdate() {
    switch (schedulerPhase) {
      case SchedulerPhase.idle:
      case SchedulerPhase.postFrameCallbacks:
        scheduleFrame();
        return;
      case SchedulerPhase.transientCallbacks:
      case SchedulerPhase.midFrameMicrotasks:
      case SchedulerPhase.persistentCallbacks:
        return;
    }
  }

当schedulerPhase处于idle状态，会调用scheduleFrame，然后经过window.scheduleFrame()中的performDispatcher.scheduleFrame()去注册一个VSync监听

 void scheduleFrame() {
    ...

    window.scheduleFrame();
    ...
  }

4. Summary

Flutter mainly passes through the screen from startup to displaying the image: firstly, it listens to the events of the window object, packages these event processing into a Framework model for distribution, creates an element tree through widgets, and then renders through scheduleWarmUpFrame, and then lays out and draws through Rendererbinding. Finally, by calling ui.window.render(scene)Scene, the information is sent to the Flutter engine, and the Flutter engine finally calls the rendering API to draw the image on the screen.

I roughly sorted out the timing diagram drawn by the Flutter view, as follows

3.Flutter performance monitoring

After having a certain understanding of view drawing, think about a question, how to control performance and optimize performance in the process of view drawing. Let’s take a look at the two performance monitoring tools that Flutter officially provides us.

1.Dart VM Service

1.observatory

Observatory: You can find its specific implementation in engine/shell/testings/observatory. It opens a ServiceClient to obtain the running status of dartvm. When the flutter app starts, it will generate the address of the current observatory server.

flutter: socket connected in service Dart VM Service Protocol v3.44 listening on http://127.0.0.1:59378/8x9XRQIBhkU=/

For example, after selecting the timeline, performance analysis can be performed, as shown in the figure

2.devTools

devTools also provides some basic inspections, and the specific details are not as complete as Observatory provides. The visibility is relatively strong

It can be installed by the following command

flutter pub global activate devtools

After the installation is complete, open it through the devtools command and enter the DartVM address

The page after opening

The timeline in devtools is performance. After we choose, the page is as follows, the operation experience is much better

Both observatory and devtools are implemented through vm_service. There are many online guides, so I won’t go into details here. I will mainly introduce Dart VM Service (hereinafter referred to as vm_service).

It is a set of web services provided inside the Dart virtual machine, and the data transmission protocol is JSON-RPC 2.0.

However, we don't need to implement data request parsing by ourselves. The official Dart SDK has written a usable Dart SDK for us to use: vm_service . When vm_service starts, it will start a WebSocket service locally, and the service URI can be obtained in the corresponding platform:

1) Android is in FlutterJNI.getObservatoryUri() ;

2) iOS is in FlutterEngine.observatoryUrl .

After we have the URI, we can use the service of vm_service. The official SDK has been written for us: vm_service

 Future<void> connect() async {
    ServiceProtocolInfo info = await Service.getInfo();
    if (info.serverUri == null) {
      print("service  protocol url is null,start vm service fail");
      return;
    }
    service = await getService(info);
    print('socket connected in service $info');
    vm = await service?.getVM();
    List<IsolateRef>? isolates = vm?.isolates;
    main = isolates?.firstWhere((ref) => ref.name?.contains('main') == true);
    main ??= isolates?.first;
    connected = true;
  }

  
  Future<VmService> getService(info) async {
    Uri uri = convertToWebSocketUrl(serviceProtocolUrl: info.serverUri);
    return await vmServiceConnectUri(uri.toString(), log: StdoutLog());
  }

Get the frameworkVersion, call the callExtensionService of a VmService instance, and pass in'flutterVersion' to get the current flutter framework and engine information

 Future<Response?> callExtensionService(String method) async {
    if (_extensionService == null && service != null && main != null) {
      _extensionService = ExtensionService(service!, main!);
      await _extensionService?.loadExtensionService();
    }
    return _extensionService!.callMethod(method);
  }

Get memory information, call getMemoryUsage of a VmService instance, you can get the current memory information

  Future<MemoryUsage> getMemoryUsage(String isolateId) =>
      _call('getMemoryUsage', {'isolateId': isolateId});

To get the FPS of the Flutter APP, the official provides several ways to let us view the performance data such as fps in the process of developing the Flutter app, such as devtools. For details, see the documents Debugging Flutter apps, Flutter performance profiling, etc.

// 需监听fps时注册
void start() {
  SchedulerBinding.instance.addTimingsCallback(_onReportTimings);
}
// 不需监听时移除
void stop() {
  SchedulerBinding.instance.removeTimingsCallback(_onReportTimings);
}
void _onReportTimings(List<FrameTiming> timings) {
  // TODO
}

2. Crash log capture and report

There are two main aspects to flutter's crash log collection:

1) Exception of flutter dart code (including app and framework code, generally will not cause a crash, guess why)

2) The crash log of the flutter engine (generally it will crash)

Dart has a Zone , which is somewhat similar to the meaning of sandbox Different zone code contexts are different and do not affect each other. Zones can also create new subzones. Zone can redefine its own print , timers , microtasks and the most critical how uncaught errors are handled.

runZoned(() {
    Future.error("asynchronous error");
}, onError: (dynamic e, StackTrace stack) {
    reportError(e, stack);
});
 

1. Flutter framework exception capture

Register the FlutterError.onError callback to collect exceptions thrown outside the Flutter framework.

FlutterError.onError = (FlutterErrorDetails details) {
    reportError(details.exception, details.stack);
};

2. Flutter engine exception capture

The exception of the flutter engine part, taking Android as an example, is mainly an error occurred in libfutter.so.

This part can be directly handed over to the native crash collection SDK for processing, such as firebase crashlytics, bugly, xCrash, etc.

We need to pass the dart exception and stack to bugly sdk through MethodChannel.

After collecting the exception, you need to check the symbol table (symbols) to restore the stack.

First, you need to confirm the version number of the flutter engine, and execute it on the command line:

flutter --version
The output is as follows:

Flutter 2.2.3 • channel stable • https://github.com/flutter/flutter.git
Framework • revision f4abaa0735 (4 months ago) • 2021-07-01 12:46:11 -0700
Engine • revision 241c87ad80
Tools • Dart 2.13.4

You can see that the revision of Engine is 241c87ad80.

Secondly, find the symbols.zip corresponding to cpu abi on flutter infra and download it. After decompression, you can get the debug so file with symbol information-libflutter.so, and then upload the stack according to the platform document to restore the stack, such as bugly The platform provides upload tools

java -jar buglySymbolAndroid.jar -i xxx

4. Flutter performance optimization

In business development, we must learn to use devtools to detect engineering performance, which will help us achieve more robust applications. During the troubleshooting process, I found that the video details page has a time-consuming rendering problem, as shown in the figure.

1.Build time-consuming optimization

The build time of the VideoControls control is 28.6ms, as shown in the figure

So our optimization plan here is to improve the build efficiency, reduce the starting point of Widget tree traversal, and distribute the setState refresh data to the underlying nodes as much as possible, so the subcomponents that trigger the refresh in the VideoControl are extracted into independent Widgets, and setState is issued to the extraction. Inside the Widget

After optimization, it is 11.0ms, and the overall average frame rate has reached 60fps, as shown in the figure

2. Paint time-consuming optimization

Next, analyze whether there are any parts that can be optimized in the paint process. We can open the debugProfilePaintsEnabled variable analysis to see the paint level displayed by Timeline, as shown in the figure.

We found that the frequently updated _buildPositionTitle and other Widgets are in the same layer. The optimization point we think of here is to use RepaintBoundary to improve paint efficiency. It provides a new isolation layer for content that frequently changes. The new layer paint will not Affect other layers

Take a look at the optimized effect, as shown in the figure

3. Summary

In the Flutter development process, when we use devtools to troubleshoot and locate page rendering problems, there are two main points:

1. Improve build efficiency , setState refresh data is sent to the bottom node as much as possible.

2. Improve paint efficiency , RepaintBoundry creates a separate layer to reduce the repainting area.

Of course, performance tuning in Flutter is far more than just this case. Each process of build / layout / paint actually has many details that can be optimized.

5. Summary

1. Review

This article mainly introduces the technology of Flutter from three dimensions, respectively explaining the drawing principles. We reviewed the source code and found that the entire rendering process is a closed loop. Framework, Engine, and Embedder each perform their duties, in short, it is Embedder. Keep getting back the Vsync signal, and the Framework will hand over the dart code to Engine to translate it into cross-platform code, and then call back to the host platform through Embedder; performance monitoring is to constantly insert our sentinels in this cycle, observe the entire ecology, and obtain abnormal data reports; Performance optimization Through a project practice, learn how to use tools to improve our efficiency in locating problems.

2. Advantages and disadvantages

advantage:

We can see that Flutter has formed a closed loop in the view drawing process, and the two ends have basically maintained consistency, so our development efficiency has been greatly improved, and performance monitoring and performance optimization are also more convenient.

shortcoming:

1) Declarative development of dynamic operation view nodes is not very friendly, it cannot be programmed like native imperative, or as easy as obtaining dom nodes from the front end

2) Realize the dynamic mechanism, there is no better open source technology to learn from