With the first real world usage of Kyanite
came the first bugs. One of the biggest bugs was an apparent memory leak. Part of it was coming from an extraneous image query code that I no longer needed so removed rather than fixed. However most of it was coming from the unlikeliest place. There are a few common text widget types but the two I was using were: Text
and SelectableText
. SelectableText
was used anywhere I wanted users to copy/paste text of posts/comments/etc. It turns out that was the source of not only my memory leak but some performance problems as well.
NOTE: Because of how responsive the Flutter team is there is already a patch in pull request but I wanted to explore this here anyway. The article shows how ListViews are supposed to operate, how one builds performant ListViews with builders, etc. Also until the fix does go into a mainline version it will be important people keep this caveat in mind with SelectableText widgets.
Problem Statement
I noticed this a bit in my initial testing but during demonstrations and with some real world use it was clear that there was a problem somewhere in the implementation of the Posts view at least, shown below:
Everything would start off fine but the longer one scrolled down the slower the process became. Looking at the memory usage according to the OS it was growing in perpetuity. The bigger problem would be when I user grabbed the scroll bar and tried to scroll all the way down. Unlike on Facebook where it feeds you pages of data about 10 posts at a time, Kyanite has your entire history. So the bottom of the scroll bar isn’t some arbitrary dozen or so posts but literally everything the user had ever written. Still I wasn’t expecting the app to freeze as it tried to power through the list. Worse, in real use the app would crash when it ran out of memory. Something was wrong there. The problem isn’t the list having thousand and thousands of items, or at least it shouldn’t be. But in this case sure enough at best performance degraded and at worst the app crashed with out of memory errors.
Diagnosis
Let’s look at how a ListView is supposed to work in Flutter, and how they work in any UI system with reasonable performance. A simplistic way of handling list views is to get all your data and shove it into the list:
ListView(
children: [
listViewItem1,
listViewItem2,
...
listViewItemN,
],
)
This works great for smaller collections, on the order of a couple to a few dozen maybe. However in our case with thousands to tens of thousands of items it would create a lot of work and drag the UI performance down. Most performant UIs have a way of not trying to render UI elements for all list items but instead just the ones it needs to properly render the UI and maybe a few on either side. In some languages this happens by passing a template that is used for mapping the underlying data to the UI at render time. In Flutter, like in SwiftUI, this sort of separation doesn’t exist. Instead there is a “builder” construct which operates like templates in the other languages. Instead of passing back a ListView widget you pass back a ListView builder which given a specific index knows how to render that one underlying data object:
ListView.builder(
itemCount: data.length,
itemBuilder: (context, index) {
return Text('${data[index].toString()}');
}
)
What happens then is Flutter on demand builds the ListView objects as it needs to and not before. An ever better version is ListView.separated
which is identical to the above but allows producing separators between the elements as well. In practice what happens is that Flutter not only creates these items on demand but it also limits the total number of them. Looking at the application I created to diagnose the problems, it had a ListView with about 27 visible items:
Looking at the object management done for this with 10,000 objects for one run we see that at no time are there more than 54 instances of the ListView items. Literally at the millisecond level or faster Flutter is creating and destroying the items as they need to to keep the number of objects reasonable but prefetching to make the UI seem more seamless (both a larger view and zoomed into 2 seconds of the data).
By keeping up with the object management Flutter stays performant and memory use stays constrained. Unfortunately that is not what was happening in Kyanite, nor with the diagnostic app, with the SelectableText objects. By simply replacing the Text objects in the diagnostic app with SelectableText objects the number of ListView items goes from a steady ~50 to an ever increasing number:
Flutter is no longer actively maintaining a reasonable number of objects by generating needed ones on the fly and deleting old ones but instead is holding on to all of them. This causes a massive explosion in objects being managed. That not only starts killing memory, even this simple app quickly starts needing many hundreds of MB, but also kills performance too. This automatic traversal of the entire list of objects ends up taking seven times longer in this scenario. In the real app the practical reality was the app crashing due to running out of memory.
In experimenting with various means of scrolling through the list it seems that the only time any ListView item that have any SelectableText item in its tree structure is deleted is when there is a discontinuous jump in the timeline by overly aggressive scrolling. Below we see the graph of this behavior. It was created by grabbing the scroll bar cursor and dragging it rapidly, but somewhat continuously, down from the top all the way to the bottom. Below we see the ListView object counts for both the case with Text and SelectableText objects as ListView items.
In the case of the Text widgets it is trying to but struggling to keep the ListView object count in the same ~50 range. We are really stressing it by dragging it rapidly so sometimes it falls behind. Those spikes though are only a few milliseconds long though. The frame rate of the UI is affected but it still stays around 30 FPS minimum. The SelectableText case however still sees the object count climb. There are some disposals going on though. You see that in the plateau region. Why they are induced though I can’t say. Only in extremely rapid scrolling does it happen. Importantly though the UI performance degrades rapidly. The collection operations are large and cause a multi-second freeze of the application. The scroll performance also degrades as we get further down the list.
Cause
The heart of the problem appears to be the SelectableText’s use of the AutomaticKeepAliveClientMixin. This was causing referential behavior which meant that the objects were never clear to be disposed. Why were any ever disposed? I’m not exactly sure. However the behavior was causing unrelated issues with NestedScrollView and Tab widgets,documented in this GitHub issue . It seems that this particular side effect with ListViews wasn’t documented yet so I added my data to it.
Solution
The solution to the problem appears to be to change the way SelectableText decides whether it should be kept alive or not. According to what Justin McCandless discovered the intended behavior was that SelectableText is kept alive while it has focus. The rest of the time there is no need for that. Within less than a day of me adding to the original issue he already had this pull request fixing it! This is a great testament to how much active interactive development the Flutter team is doing and that their issue tracker is not just a repository for the external community to track problems which are then not looked at frequently.
While this fix will be great once it is in a mainline release in the interim people should be aware of this behavior and code accordingly. In my particular case I think the mitigation works better than my original for Kyanite. In the original version if one wanted to get text from a post, etc. they had to click/drag and then right-click to copy. That is assuming they realized that was a possibility. I was going to add documentation to that effect. Now there is a very easy to find “Copy” button which not only copies the text but actually dumps the whole thing to the clipboard so it can be used elsewhere:
As you can see the button highlighted with the red circle is new. If one hits it they get all of the data and metadata about the post that is available:
Title: Timeline Photos
Creation At: July 19 2021 9:34 PM
Text:
Watching the classic movie Reds for the first time and this one interviewee has glasses with a corrective power I have never seen before except done for comedy (like Milton in Office Space but even more powerful).
Photos and Videos:
/home/hankg/facebook-hankg/posts/media/TimelinePhotos_5LWaouljMq/OWRjNDU3YTVlYjM2Njk0MjUzOWY2ZTFi.jpg
/home/hankg/facebook-hankg/posts/media/TimelinePhotos_5LWaouljMq/NzVkOTEwOTEwM2EzZTMwMjg0ZDNlOWNl.jpg
/home/hankg/facebook-hankg/posts/media/TimelinePhotos_5LWaouljMq/YWM0NDE1YmM0ZTNlMmE5YTAzZjIxOGZj.jpg
So while I’m glad it will be fixed I think the new UI is going to stay the way it is. In the mean time I’ll begin testing the fix to confirm the behavior.
Source Code
The source code used for generating this data can be found below:
import 'package:async/async.dart';
import 'package:flutter/material.dart';
final initialTime = DateTime.now().millisecondsSinceEpoch;
bool useSelectable = false;
int count = 1000;
void main(List<String> args) {
if (args.length >= 1) {
useSelectable = args[0].toLowerCase() == 'true' ? true : false;
}
if (args.length >= 2) {
count = int.tryParse(args[1]) ?? 1000;
}
print('Use SelectableText = $useSelectable');
print('Count = $count');
runApp(const MyApp());
}
class MyApp extends StatelessWidget {
const MyApp({Key? key}) : super(key: key);
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'Flutter Demo',
theme: ThemeData(
primarySwatch: Colors.blue,
),
home: const MyHomePage(title: 'Flutter Demo Home Page'),
);
}
}
class MyHomePage extends StatefulWidget {
const MyHomePage({Key? key, required this.title}) : super(key: key);
final String title;
@override
State<MyHomePage> createState() => _MyHomePageState();
}
class _MyHomePageState extends State<MyHomePage> {
int currentIndex = 0;
int speedMS = 50;
double height = 20.0;
RestartableTimer? timer;
ScrollController listViewController = ScrollController();
void forward() {
if (currentIndex < count) {
setState(() {
currentIndex++;
listViewController.jumpTo(currentIndex * height);
});
}
}
void startTimer() {
timer ??= RestartableTimer(Duration(milliseconds: speedMS), nextSlide);
timer?.reset();
}
void stopTimer() {
timer?.cancel();
}
void nextSlide() {
if (currentIndex == count - 1) {
stopTimer();
}
forward();
timer?.reset();
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text(widget.title),
),
body: Center(
child: Column(
mainAxisAlignment: MainAxisAlignment.start,
children: <Widget>[
Text(
'$currentIndex of $count lines',
),
Row(
mainAxisAlignment: MainAxisAlignment.center,
crossAxisAlignment: CrossAxisAlignment.center,
children: [
ElevatedButton(onPressed: stopTimer, child: Icon(Icons.stop)),
_buildListView(context),
ElevatedButton(onPressed: startTimer, child: Icon(Icons.arrow_forward)),
],
),
//_buildListView(context),
],
),
),
);
}
Widget _buildListView(BuildContext context) {
return SizedBox(
width: 600,
height: 600,
child: Padding(
padding: const EdgeInsets.all(8.0),
child: ListView.separated(
controller: listViewController,
itemBuilder: (context, index) {
return TextWidget(index: index, height: height);
},
separatorBuilder: (context, index) {
return const Divider(height: 1);
},
itemCount: count),
));
}
}
class TextWidget extends StatefulWidget {
final int index;
final double height;
const TextWidget({Key? key, required this.index, required this.height}) : super(key: key);
@override
State<TextWidget> createState() => _TextWidgetState();
}
class _TextWidgetState extends State<TextWidget> {
@override
void initState() {
print('${widget.index},${DateTime.now().millisecondsSinceEpoch-initialTime}, 1');
super.initState();
}
@override
void dispose() {
print('${widget.index},${DateTime.now().millisecondsSinceEpoch-initialTime},-1');
super.dispose();
}
@override
Widget build(BuildContext context) {
final text = 'List line ${widget.index}';
return SizedBox(
width: 600,
height: widget.height,
child: useSelectable ? SelectableText(text) : Text(text),
);
}
}