The Joke that is UAT

clown shoes


One of the final stages of development tends to be UAT – User Acceptance Testing, where end users are presented with the final product and asked to test and sign off on the product.

In a recent session it struck me how lubricious it was to combine acceptance testing with system testing. Expecting an end user to instantly identify not just UX problems, but formula problems in a very fast half hour meeting. Without being intimately familiar with the underlying data how could they spot all issues – spotting 5.0% when it should be 7.4%.

System testing is taken care of by Unit Tests and ATDD in the hopes that any issues with the underlying logic will be exposed and corrected. These should be based on the requirements that the team received, and these should have been signed off by the business sponsor.


Course what we tend to find is that users submit requests that are more similar to wish lists than well defined requirements. Now as developers work on these requirements, we start creating unit tests based on our interpretation of the requirements. If there is disagreement on the team about the interpretation, then we seek the business user to confirm.

Yet if there is no disagreement on the interpretation then the formulas get implemented and the first time the user gets a chance to spot the issue is during UAT. Now we have a bug in the system that got full sign off from everyone and will remain in the system until someone notices the difference.

Computer Bug

So now the question is how could we have seen this coming and better prepared to prevent last minute changes or introducing bugs…


The old encapsulation diet is back in fashion

Stack of hats

A major coding principle deals with encapsulation – the basic idea is you want two services to chat with the minimum number of dependencies, this is known as loose coupling.

A way to think of this is at a restaurant – the waitress takes your order and she needs to know order number and the customers, she hands off the order to the cook who prepares the meal. The cook does not need to know anything about the customers (ok, so many allergies, special requests… :)). This information, the customers, is encapsulated by the waitress!

Each level of the networking protocol stack encapsulate the data of the layers above it. We can look at the four layer network stack:



This provides a huge benefit in terms of design, allowing each layer to be programmed separately. When a message is received by a layer it is broken up into two sections – the header and the payload.

The payload contains all the data for the layers above our layer, but the header is constructed by the corresponding layer on the sender’s machine. The header lays out the information that is used by the layer to take whatever action is required. For instance the IP layer contains the destination IP address, or the transport layer stipulating the port.

We do not need to know about the logic in the layers above and below – if data is coming from above, we simply add it all into the payload section. This loose coupling means network stack can be constructed by separately developed layers, and upgraded independently of everything else.

Think about when you have your laptop – if you have a network cable plugged in, then the physical layer uses the correct 802.3 network protocol, while if it is using a wireless card then it will implement the 802.11x protocol. The layers above do not need to know how the data is being transmitted, as such when another method becomes available then only the physical layer needs to be upgraded.

802.11 network stack


So, the stack is all about encapsulation!

Multiplexing of the Past

I finally understand the old problem of trying to balance accessing the internet and leaving the telephone line open for phone calls, and how it was eventually resolved.

The basis of the problem is the fact that there is a single physical copper line coming into the household. With a single line we were limited to sending information from one end to the other – think of the switchboard operator who would have to manually connected two lines!

Telephone_operators,_1952Eventually the operator was put out of a job by automatic exchanges which provided the first dial tone. Still we had the problem that a single line could only be used to transmit a single signal at a time.

In comes multiplexing! We have a limited resource, the telephone line, which we want to share so the concept of multiplex was created where the shared resource was split up but everyone who used it had no idea others might also be using the line.

time division multiplexing

There are many types of multiplexing:

  • Synchronous Time-Division Multiplexing (STDM)
  • Frequency-Division Multiplexing (FDM)
  • Statistical Multiplexing

STDM is one of the more basic ideas, where time is simply broken up and assigned in a round-robin style to everyone who wants to use the line, i.e. so maybe I only use the line every X milliseconds.

FDM is slightly more complex and something you will recognize as one of the solutions implemented so us nerds could continue to connect to the internet. The idea is that your voice and hearing has a very specific range, outside of that you can’t hear – think dog whistle.



So the brilliant idea was that there is a huge amount of frequencies that are being left out, and that if we transmitted our packets in another bands, that we could actually transmit both together. So again we would divide up the allocated frequency band by the number of users who wanted to use it and restrict them to send on those.

When DSL broadband came in you needed to go around to all the phones in the house and add a DSL filter to the line, to split out these frequencies to ensure as the phones would not know what to deal with all this extra information coming in at these higher frequencies.

dsl filter

This works up to a the point you want to scale the system, which is why we have have statistical multiplexing. This is based off STDM, but sets an upper bound on the the time it is allocated and is the origin of the network packet! Now messages would be broken up into these specific sized network packets and at the line each packet is assessed on a packet-by-packet basis to prioritize them.

This is all the responsibility of the physical layer in the network protocol stack, though it is only one of many of it’s responsibilities! More to come…

Android and Robotium (To the rescue!)



Ah, I love a good regression suit that passes after check-in!

This is where Robotium is really making a big difference for me especially as I try and adhere to Acceptance Test Driven Development, ATDD. The basis of ATDD is designing a test around the acceptance criteria/success measures of a story/task/use-case. This is done before development and once you have it passing, that is it – development is done (ok.. plus any cleanup, re-factoring..)!

Given a story, we break this down into the individual tasks the user takes and each task will have an ATDD test. Our story is:

As a User,
I want to find a bar,
So that I can see it's menu

When we break this story down into its individual tasks we get the following list:

1. User clicks “Find Bar” button from the main menu

2. User enters bar name into input field and selects “Search”

3. User is returned the bar’s menu

So each of these becomes a test case which starting off all fail:

package com.robotium.test;
import com.robotium.solo.Solo;
import com.balortech.MainActivity;
import android.test.ActivityInstrumentationTestCase2;
public class BreezeTest extends ActivityInstrumentationTestCase2{
	private Solo solo;
	public BreezeTestTest() {
	public void setUp() throws 1.5.0/docs/api/java/lang/Exception.html">Exception {
		//setUp() is run before a test case is started. 
		//This is where the solo object is created.
		solo = new Solo(getInstrumentation(), getActivity());
	public void tearDown() throws 1.5.0/docs/api/java/lang/Exception.html">Exception {
		//tearDown() is run after a test case has finished. 
		//finishOpenedActivities() will finish all the activities that have been opened during the test execution.
	public void testMainMenuFindBar() throws 1.5.0/docs/api/java/lang/Exception.html">Exception {
		assertTrue("Main Menu's Find Bar button sends the user to the right activity", false); 
	public void testFindBarSearch() throws 1.5.0/docs/api/java/lang/Exception.html">Exception {
		assertTrue("Entering a bar's name in the search field send the user to the results", false); 
	public void testBarMenu() throws 1.5.0/docs/api/java/lang/Exception.html">Exception {
		assertTrue("The bar's result page shows up correctly", false); 

Then taking the first test we start to build out it out:

public void testMainMenuFindBar() throws 1.5.0/docs/api/java/lang/Exception.html">Exception {
		solo.clickOnMenuItem("Find Bar");
		//Assert that NoteEditor activity is opened
		solo.assertCurrentActivity("Main Menu's Find Bar button sends the user to the right activity", "FindBar"); 

At this point we leave the ATDD and begin building up our Unit Tests and the code to drive those – again the Unit Test written first, implement the java to pass it and so on. At the end when the final Unit Test passes, the ATDD test should pass. Then on to the next one!

Happy testing folks ;)

Baby Steps with Android

Screen Shot 2014-03-10 at 11.13.25 PMAh that glorious moment where everything finally works!

One of the major changes in direction we are taking with BalorTech is to focus on the journey and build an MVP around each step rather than develop pieces independently and hooking them together later.

As such our first step was actually setting up an Android application that presented a user with the button to Find Bars, which when pressed will call an web service.

A lot of time has been spent on trying to figure out the normal testing framework for Android – both a white box and black box. It appears that Android provides a number of testing frameworks with their development toolkit ranging from monkeyrunner to uiautomator. There are some external options also in the form of Robotium which overwrites default Android classes on run time. More analysis will be done to see which would be best to implement for ATDD…