App development, Xamarin Test Cloud & Continuous Integration - Part 1
This summer, starting our HeartStarter v.2.0 project, a Xamarin.Forms mobile app for both iOS and Android, we knew we wanted to improve our workflow and have better quality assurance. Thus we decided to start writing and submitting our UI tests to Xamarin Test Cloud, and setting up continuous integration for all our apps. These topics will be covered in two blog posts: Part 1 - How writing, running and submitting UI tests to Tamarin Test Cloud, and a Part 2 - Setting up Continuous Integration for your Apps.
Writing, running and submitting UI tests to Tamarin Test Cloud
After reading Part 1, you will be able to write and execute your first tests and validate the functionality of your iOS and Android Apps by running it on hundreds of devices in Test Cloud.
Introduction to Xamarin.UITest - Automated Mobile Testing with C# and NUnit
As described by Xamarin in their "Introduction to Xamarin.UITest" guide, Xamarin.UITest is a testing framework that enables Automated UI Acceptance Tests written in NUnit to be run against iOS and Android applications. It integrates tightly with Xamarin.iOS and Xamarin.Android projects but it can also be used with iOS and Android projects written natively in Objective-C and Java. Xamarin.UITest is the Automation Library that allows the NUnit tests to execute on Android and iOS devices. The tests interact with the user interface just as a user would, enter text, tapping buttons, and performing gestures – such as swipes.
Writing your first test
Typically, each UITest is written as a method that is referred to as a test. The class which contains the test is known as a test fixture. The test fixture contains either a single test or logical grouping of tests and is responsible for any setup to make the test run and any cleanup that needs to be peformed when the test finishes. Each test should follow the Arrange-Act-Assert pattern:
- Arrange – The test will setup conditions and initialize things so that the test can be actioned.
- Act – The test will interact with the application, enter text, pushing buttons, and so on.
- Assert – The test examines the results of the actions performed in the Act step to determine correctness. For example, the application may verify that a particular error message is displayed.
The code to be tested:
For this demo, we have a small app that presents the user with a button that says "Click me!". When clicking the button, it changes the text to "x clicks!", where x represents the number of clicks.
Lets take a look at the actual code for achieving this.
Writing our UI test:
Running our test in the IDE:
Click "Run all" in the test-panel, this will automatically launch your app and run the given test. The screenshot below shows the test being run through the iOS simulator.
Yay, our test passed!
But we want to make sure this works not only in our simulator, but on an actual device, on the most commonly used devices, so we can assure our users are having the same experience using our app as we have.
Submitting our tests to Xamarin Test Cloud
Both Visual Studio and Xamarin Studio can submit UITests to Xamarin Test Cloud when the solution contains both a mobile Xamarin project and a UITest project. When UITests are submitted to Xamarin Test Cloud, the IDE will rebuild the application and the tests and upload the binaries, placing them in a queue for execution.
- Before submitting an iOS project, you must ensure that your build configuration is set to Debug > iPhone and that the iOS project is set as the startup project.
- Next, right click on the project in the Test Panel, and select Run in Test Cloud.
-
When the Run in Test Cloud dialog to appear. Select the iOS version:
-
Your app is now being uploaded to Test Cloud:
-
When done, your browser will launch and redirect you to testcloud.xamarin.com. Here you will be presented with the following screen where we'll choose to run our tests on "the top 10 phones" for this demo:
-
The tests will now be queued in Test Cloud and you can see the tests being processed:
-
When the test run is finished, Test Cloud will send out an e-mail with a summary of the test results and you can click the report to view more details about the tests:
-
Viewing the test details you also get the option of viewing results for a given device:
-
Here you see test results for the iPhone 6:
That's it!
Happy testing and remember to read my next blog post where I will teach you how to set up continuous integration for your app using Team City!