The Key to Widget Testing
An in-depth look at Testing in Flutter.
Unfortunately, I inadvertently deleted the source code for this article. It’s a good read, but without the code, it’s not stellar. Give it go, and I’ll have another example app to work with one day.
You’ve got to test your code. It’s a given. You’ve got to make sure it works, but like documentation, we programmers tend to write our test cases almost as an afterthought. Some shops, however, dedicate whole departments just to testing. Someone else then tests your code! Flutter offers a whole platform just for testing your code. Let’s take a look at how it’s done.
There’s A Test For That
In every new Flutter project, you‘ll find a directory called, test. In this article, we’re going to look at the test file, widget_test.dart, found in that directory for this particular Flutter project below called, mvc_template. You’ll have access to the Github account for this project (Again, I mistakenly deleted this project). It’s to produce a ‘project template’ involving the MVC design pattern for developers to start their own apps, but that’s another story. Here, we’re testing its accompanying example app.
I Like Screenshots. Tap Captions For Gists.
As always, I prefer using screenshots in my articles over gists to show concepts rather than just show code. I find them easier to work with frankly. However, you can click or tap on their captions to see the code in a gist or in Github. Tap or click on the screenshots themselves to zoom in on them.
No Moving Pictures, No Social Media
There will be gif files in this article demonstrating aspects of the topic at hand. However, it’s said viewing such gif files is not possible when reading this article on platforms like Instagram, Facebook, etc. They may come out as static pictures or simply blank placeholder boxes. Please, be aware of this and maybe read this article on medium.com
Let’s begin.
A Test Run
The screenshot below is the test file, widget_test.dart. On its left is a gif file demonstrating some of the functions and features of the example app while on its right is a gif file depicting what happens when that test file runs. You can see the test run goes through most of the app’s functionality, and if all goes correctly, the test run reports its series of tests have passed in your IDE’s console screen. Otherwise, the run will stop abruptly when it first encounters an error. Note, going through the app’s functions and features programmatically, you’ll find the whole process goes by in a flash. However, we’ll walk through this test file, today, and give you some insight as to how you can write such test files for your own code.
The Source Is Key
Looking at the little red arrows above, let’s start with the concept of ‘unit testing’ as it’s described in the Flutter documentation, An introduction to unit testing. For example, you can see in the test file, the statement, con = Controller();, involves a class right out of the example app’s own source code. When testing, not only are you able to start up your app with the usual command, tester = pumpWidget(MyApp());, you’re able to perform the direct testing of individual classes and modules right in the code since you’ve likely access to all of the app’s source code.
If however there isn’t a pressing need for unit testing as such, this extensive access also allows you to orchestrate how the testing is to go about. This in itself could be even more important. For example, the highlighted class members, App.useCupertinoand, con.wordsApp, allows you to implement a flow of control and, in this case, control the execution of particular tests.
This example app, for instance, has the ability to toggle back and forth between the Cupertino interface design and the Material design, and that ability is to be tested here. The ‘if’ statement, if(App.useCupertino), will have the app switch to Material design before testing begins — it’s getting the ‘test environment’ ready before proceeding.
As for the ‘if’ statement, if(con.wordsApp), the app also runs two separate apps: The startup app (counter app) or the Random Words (Word Pairs) app. Which one depends on the boolean value of the instance field, wordsApp. Note, you’ll soon discover this test file is made up of a collection of high-level functions I’ll call Test functions — each dedicated to testing a particular aspect of the example app. Introducing such functions makes the test file’s code organized and modular.
A Singleton Approach
Let’s note the ‘Controller’ object instantiated in the test file (first arrow above), is a very important player in this whole enterprise as it’s part of the MVC design pattern used in this example app itself. In particular, it’s important to note it also follows the singleton design pattern instantiating only one instance of this class for the duration of the application’s runtime. The screenshot below on the left-hand side confirms this with its use of a factory constructor. It was determined the role of such a class only needs one instance, and such a fact further benefits the app’s testing because you’re then free to instantiate this class in the test file to help with the testing.
Further still, the static field, App.useCupertino, comes from the underlining MVC framework utilized by the example app. Since the test file has access to the app’s source code, it stands to reason it also has access to its underlining framework — further assisting in the testing.
Import Your Source
A screenshot of the beginning of the test file reveals the import statements involved in making all this possible. There’s nothing new here. They’re the very same import statements used in the example app itself. Introducing them here as well merely makes your testing that much more effective and thorough. For example, it was only the ‘view’ import statement (listed second last) that starts up the app’s main class, MyApp. The other two adds the flow of control capability introducing the members, App.useCupertinoand, con.wordsApp.
The System Knows
By the way, those class members referenced so far, are accessing the app’s system preferences — the interface design and the running app used at any one time are recorded under preferences. This is so their settings are chosen again the next time the app starts up. As is the case, the test file also needs to know the last chosen preference so as to run properly. Hence its access to those class members, App.useCupertinoand, con.wordsApp. The expression,con.wordsApp, is a getter determining the boolean value from the system’s preferences. See below. The class, Prefs, just happens to be part of the underlining framework used by the example app.
Make The Switch
Let’s now examine the first Test function we would possibly run in this test file. It’s when the App is currently running in Cupertino design, and we want to therefore switch over to the Material design interface before proceeding. This will involve the high-level function called, openInterfaceMenu, passing in the WidgetTester object, tester, to accomplish specific testing. It will attempt to open the example app’s popup menu and select the menu option labeled, interface. If it fails, the whole test file stops abruptly then and there.
The screenshot below presents the test function involved. However, the first test function listed is the openPopupMenu() function. The popup menu has to be opened and be the topmost focus so as to then tap on the menu option, Interface. You can see how that’s all done below. The expect() function is used extensively to confirm the test is proceeding as…well,...expected.
Using Keys Is Key
Note in the test functions listed above, all three utilize the Key values of the widget’s sought after in each test — the find.byKey() function is used in all three Test functions. The app’s popup menu, for example, explicitly assigns a LocalKey value to its PopupMenuButton widget. See below. Further still, in the next screenshot below, the ‘interface’ menu option has a LocalKey value involved in the test function called, openInterfaceMenu. Thought I’d mention this now, but let’s move on.
This Way Or The Other
Note in the screenshot below, the code that will switch from one app to the other in the test file. It’s set up to switch from the ‘Counter app’ to the ‘Word Pairs app’, and does this by calling the very routine responsible for the operation, and not work through the interface using the test function, openApplication.
The code could have just as easily been the other way round using the openApplicationMenu() function to perform the switch. I mention this to demonstrate again the possible arrangements in a typical test case. Possibly the testers assigned to your code only know ‘the interface’ involved in such operations. They’re not given the Controller’s API for example and don’t know about the function, changeApp. On the other hand, they may fully understand the app’s API and can perform such ‘unit tests’ right then and there instead.
Instead, the very routine being tested is called directly. A screenshot of that routine is below. As it happens, this example app is designed in such a fashion to make this possible. Otherwise, the test function would have to correctly work with the interface to invoke the operations to be tested. Something to consider in any event.
What’s The Word?
The next series of tests is found in a function called, wordsTest, and is concerned with the Word Pair app. The function is listed below, and again, class objects defined from the app itself are used to determine how the tests are performed. For example, depending on whether the app is running in Cupertino or Material design, the ‘if’ statement below will look for the appropriate ‘type’ of ListTile objects used to display the many word pairs. The test ‘expects’ to find one or more of such widgets. As it happens, I decided to end the testing there if the app is running in Cupertino design (letting it only continue if in Material design), and that requires further defining the flow of control.
If in the Material design, the Test function continues by tapping on the first three listed word pairs as you can see in the gif file above. Again, the use of a key is utilized to reliably find the IconButton widget that when tapped will take us to the ‘Saved Suggestions’ screen. Originally, it did use the find.byType() function to find the appropriate widget to tap, but I’m finding the ‘byKey’ approach to be generally more reliable and adaptive.
It’s A Mix Match
Note the operation is confirmed by again taking advantage of the full access to the app’s source code. The Model object responsible for the ‘data aspect’ of the app is accessed (also Singleton approach by the way) to determine those three tapped word pairs were successfully saved. That code is surrounding by the ‘test code’ used to navigate through the app’s interface. Such code has opened the ‘Saved Suggestion’ screen and then has tapped the ‘back button’ to return. The function, pump, is used ‘to wait’ for such interface operations to complete.
Count On Keys
The next test function is one you’ll readily recognize. It’s the one supplied to you every time you create a new Flutter project used to test the Startup app. As you know, it looks for a Text widget displaying specific values (in this case either a zero or a one) and taps on the only Icon widget called, add. However, again, I’ve instead utilized the fact that you can find your widgets by their Key value. Do you see a pattern going on here yet?
This approach of course means the FloatingActionButton involved will have to have that very Key value for this to all work. As you see below, it does.
What’s On The Menu?
The last test function in the test file involves the rest of the menu options in the app’s popup menu. That test function is called, menuTest, and is highlighted below. Further below is a screenshot of the function itself revealing still further test functions listed one after the other — all nice and concise. Any of them will stop the test run in its tracks.
The Key State
Let’s get back to this ‘Key value’ business and the actual main topic for this article. Of course, as you may know, StatefulWidgets are created, destroyed, and created again repeatedly through the lifecycle of a typical Flutter app — leaving their corresponding State objects alone and thus preserving their state. However, reassigning a new Key to a StatefulWidget will actually cause its accompanying State object to be ‘garbage collected’ and re-created with the next call of its setState() function. At times that’s desired and so is important to remember.
When a StatefulWidget uses a LocalKey (or a GlobalKey), this allows its corresponding Element object to freely move around the tree (changing its Element parent) without losing state. There will be instances while adding, removing, or reordering a collection of StatefulWidgets of the same type in the ‘Widget tree’, they successfully hold their state. Keys allow for this.
Emily Fortuna wrote a great article regarding Keys called, Keys! What are they good for? It supplemented one of the many Google YouTube videos she presented as Senior Developer Advocate. Note, I believe she’s since moved on and is now pursuing an acting career: www.emilyfortuna.com.
The Key To Testing
For me, however, now more than two years in working with Flutter, the Key parameter has proven to be more useful in another important part of developing in Flutter: Testing in Flutter. I’m getting into the habit of faithfully assigning a local Key value to every widget of consequence in my apps so that I can reliably retrieve them when testing in their corresponding test files.
Look at the screenshot below. Every menu option has assigned a Key. The naming convention is somewhat rudimentary, granted, but when I’m later writing the test files for my app, I can take an educated guess as to the LocalKey used on a widget I’m about to test. Of course, I’m one lone freelancer, and I have that luxury. No doubt more formal organizations would take up a more standardized approach and list out these key values. That list would then be given to their ‘testing department’ so as to reliably test the appropriate widgets.
What You Key Is What You Get
You can see below the menu options corresponding Test functions. We take advantage of their Key values allowing us to readily tap on these menu options during our testing.
Note, there are some cases where you’re not free to assign a Key value to a particular widget. For example, when using third-party packages. The dialog window opened to list the app’s Locale options displays the option, Cancel, and it doesn’t allow for a Key to be assigned and yet it’s this option we wish to tap so to close that window. Happily, Flutter’s testing platform provides the function, widgetWithText, as a means to tap on the right widget to do just that.
Just Make A Showing
The final two test functions involve merely opening the ‘color’ and ‘about’ menu options. Each function tests that these widgets do indeed come up with a tap. Of course, there’s room to further test the color picker’s functionality for example, and test if it successfully changes the color theme of the app. However, that was not done through the app’s interface by the time of this writing. It would require, for example, that the appropriate ‘offset’ positioning be determined so as to tap on the color palette correctly.
Test Internally
In theory, however, with extensive access to the example app’s very code, again you could directly call the underlining mechanism involved and test that the ‘changing of color’ is indeed successful. For example, there exists a test unit function called, testColorTheme, in the test file that goes through the colors of the rainbow and so repeatedly testing this functionality.
In truth, it’s a bit much. I mean, you merely have to test for one occurrence when changing the app’s color theme, but this function is included to demonstrate again it’s perfectly alright to test your app by directly accessing the ‘internals of your app’ if your code is organized in such a way allowing you to do so. In this case, the code using the MVC design pattern is intrinsically organized so the ‘Controller’ aspect of the app is tasked to respond to system and user events. In the screenshot above, the Controller object is called upon to change the app’s color.
And so, in this test file, we have a dedicated module/function called, testColorTheme, that’s involved in changing the app’s color theme. Again, not really necessary, but it does make for a ‘colorful’ demonstration. Note, however, it’s actually commented out up on Github.
Catch Your Test Errors
You can see in the screenshot of Flutter’s testWidgets class below (the very class that calls your test file defined as ‘callback’) that when your test file is run, it’s wrapped in a try statement. However, there’s no catch clause!? By design, in setting up its own ‘test environment’, Flutter’s testWidgets class defines its own error handler to catch any error (test errors or otherwise).
Throw When You Want To
By design, the test is stopped with the first error. However, there’s no law the test file has to abruptly stop with every error. Of course, the idea behind ‘stopping everything’ with the first error encountered, is that the app is now likely corrupt. Any further running of the app may only cause a ‘snowball effect’ — cascading into more errors and possibly doing significant harm to external databases for example. Best to stop right here and now is the rule, and it’s a good way to go. However, for demonstration purposes, and being such a simple series of test that I believe won’t cause more spurious errors, in the end, I introduced a number of try-catch statements to our test file.
Is It Open?
Note, these modifications to the test file allows for the implementation of more flow of control. For example, the function, openPopupMenu, now returns a boolean value when it successfully opens the app’s popup menu. Now the test file is able to ‘change course’ if that function fails to open the app’s popup menu for some change in the interface for example and returns a boolean False. Further, all the code in that function that could error is now wrapped in a try-catch statement. There are now try-catch statements in most of the test functions.
Error In The End
The test file’s main routine now has two try-catch statements. With their presence, the test file no longer stops further testing with the first error encountered but continues on and only notifies you of errors when the test run is completed. It does this by calling the _reportErrors() function. Again, this approach would not be advisable if such errors only lead to more errors with possibly catastrophic consequences. However, you have that option.
Know Your Mistakes
Now with any error, its error message is concatenated to a String variable, _errorMessages. We’re now collecting any and all error messages, and at the end of the testing, if an error did occur in the tests, they’re declared with a throw command. In this approach, you can first test everything, and if there were any errors, they’re supplied to you all at once so as to then be addressed. It’s an option.
Well, there you have it. Look at that. The testing platform for Flutter allows you to write extensive and elaborate code to perform the necessary testing. It’s ok to take full advantage of your app’s own API to initiate and carry out such tests. Don’t be afraid to make mistakes or errors while developing such test code. After all, it’s to weed out any and all mistakes and errors lurking in your own app’s code — give yourself a reliable means to do so.
Cheers.
