It’s too often that app development projects are run without a solid chain/link between Requirements → Development → QA/Testing, where testing is neglected or performed manually. The key to effective QA is Automation. Autonomous testing can be achieved via unit testing for algorithms and UI (instrumentation) testing for functional testing. Traditional UI/functional testing is achieved by using Android’s UIAutomator API.
The purpose of this post is to illustrate how functional testing can be achieved using Calabash/Cucumba.
The Agile Alliance states that one of the benefits of BDD (Behaviour Driven Development) over TDD (Test Driven Development) as:
BDD offers more precise guidance on organizing the conversation between developers, testers and domain experts.
However, I do think that BDD can be used to complement TDD.
The premise for using Cucumber is to develop functional tests using natural language that all stack-holders, technical and non-technical, can understand and this fits in well with the agile software development methodologies that use User Stories to define the behaviour of the system between user and app (software). BDD translates easily from User Stories: The more comprehensive the User Stories, then more comprehensive your functional tests. Further more, BDD tests can be written by the business analyst defined the User Stories, developer or QA engineer.
Calabash enables you to write and execute automated acceptance tests of mobile apps. Calabash is cross-platform, supporting Android and iOS native apps. It is open source and free, and has a company, Xamarin, backing and developing it.
Chapter One – Calabash
Calabash/Cucumber consists of a server component and a client component. The Server component is a Ruby application that interprets the Feature and Step definition files to drive an Instrumentation Test Server that is installed on the test device/emulator.
Instructions to download and install Ruby and Calabash-Android are found here. Once you have installed Ruby and Calabash, you may proceed on to the next chapter.
Chapter Two – Features
As explained in the introduction, Calabash is based on Cucumber to deliver a framework with which to define Features ala User Stories using natural language as the test cases. To illustrate this, we’re going to use the scenario that we’re developing an Instant Messaging app and as with any project, there is a stage in the process where requirements are gathered. Let’s take this opportunity to define our first Story aka requirement:
“As a new user I want to register a new account, so that I may interact on the social network.”
Calabash Features, by convention are simple text files with the ‘.feature’ extension that reside in the ‘features/’ folder. Let’s have a look at our first Feature (register.feature):
@register Feature: Registration feature @register-successful Scenario: As a new user I want to register a new account Given I have entered an email and password When I press the Register button Then I should see a progress indictor Then I wait for a toaster message 'Success' Then I should see the roster
That’s it! We’ve defined out first Feature. There a number properties that are illustrated in out first attempt here:
– Firstly, we name/describe our feature using the ‘Feature:‘ attribute on line 2.
– Secondly, we identify our scenario using the ‘Scenario:‘ attribute (notice how we’ve used the same description as our Story declaration), followed by the test steps required. These can be considered the prerequisite(s), requirement(s) and then acceptance-criteria of the scenario.
– Lastly, we illustrate the use of @tags. We use these to mark points-of-reference.
At this point, if you tried running this feature, you’d get a bunch of Calabash error. That’s because Cucumber doesn’t know how to interpret out scenario steps. We’ll explore that in part two.