Web Testing

Testing an API

So you’re a tester right? So you can test this API for me? What? What on earth is n API?

Sounds familiar? Then read on for your quick start guide to testing an API.

An API is an Application Programming Interface used by developers to power apps, websites, widgets and other cool things. An API basically allows you to make requests for data and in return a website provides a response, usually in JSON format but XML is also common.

Before you get started the first and probably most important thing to know is that an API does NOT have a presentation layer, it will not look pretty, it will not have cute colours, and depending on which browser you are using it might look downright scary (thanks IE!). Never ever raise a bug about an API looking unstyled, well not unless you want to get laughed all the way into next week.

So how are you going to test this thing? Well the great thing about an API is it is intended to be used by someone who isn’t familiar with the internal workings of your website so they should have documentation and it should be easy to understand. Basically you use a URL and the available parameters to make up a request which you fire off in your browser address bar.

Your request will look something like the following. Be aware that API requests make of use of parameters in the URL, the first one will use a ? and all subsequent ones will use an &:

API request with one parameter:

http://api.songkick.com/api/3.0/users/{username}/events.json?apikey={your_api_key}

API request with multiple parameters:

http://api.songkick.com/api/3.0/users/{username}/calendar.json?reason=
attendance&apikey={your_api_key}

 

In return your browser will display something like:

{ "resultsPage": {
    "status": "ok",
    "page": 1,
    "totalEntries": 2,
    "perPage": 50,
    "results": {
        "event": [EVENT, EVENT]
    }
} }

So if you’re not testing how it looks then what are you testing? Probably the first thing to focus on is that the data being returned is actually correct, check that all the fields are filled in correctly and check that the data is accurate (you will probably be able to check this against the corresponding website). Domain knowledge will be key to finding bugs at this stage.

Next you’ll want to check that all the required requests are available, you should have some API documents to help you with this.  Read through them, check the examples work and then make sure everything they say can be done actually can be done.

Now we’re at the destructive stage. An API basically gives someone access to data from a website, therefore they are almost always restricted by an API key which allows the API owner to have control over who accesses the data and how frequently. Test that you can’t access anything without a valid key, make sure only the requests documented are actually available and if there is any way to post data back to the API make sure you give it a very thorough bashing.

Finally you should check for error handling. Just as a website has the ability to return a 404 page or maybe a 500 error when something goes wrong an API should be returning the correct error code for unavailable servers, invalid requests, empty data sets etc.

Help, there are too many web browsers to test

Over the past 4 or 5 years the web has become hugely advanced, and more and more users are accessing the web on mobile devices, all of this has led to new, more complex, web browsers and seemingly endless browser and operating system (OS) combinations. If you test a website or mobile site then you’ll need to consider where to focus your test effort, unless you have a very large budget (or a very simple site) then it is simply impossible to cover every combination in-depth so what should you do?

There are two things to consider when approaching browser/OS testing – the first is usage, what are your users using? Ideally you’ll have access to some user stats (analytics) to tell you which browsers and OSs your users are using. If not, then you’ll need to do some research, at the time of writing Internet Explorer (IE) 8 is the most widely used browser with IE9 in hot pursuit, but Safari (particular on mobile devices) and Chrome are both popular. Make sure you consider who your audience is, teenagers will be using different systems to tech savvy pensioners, which means that depending on your audience you either care deeply about Opera or you don’t care a cent, maybe most of your users are on Macs or maybe they’re on Windows,. Don’t forget to consider mobile devices, games consoles and TVs in your research and keep an eye on browser upgrade dates and OS launch dates to help you predict which combinations are relevant to you.

The second thing to consider is risk, based on your experience which browser/OS combinations will cause the most issues? This is very much an experience based metric, consider where you have previously found bugs and see if the risk applies here. If you’re new to the project then look through the bug tracking system to see what bugs have been previously fixed and consider if they could occur again, or maybe the architecture is similar to a previous project giving you an idea of how much you need to worry about cookies, cache and security settings. Some knowledge of how the website and app has been built will give you a huge help here so if you’re not sure then it could be time to have a chat with a developer.

So once you have your data you can start to put together a prioritised list of the browser and OS combinations that need testing. It is a very good idea to run through all your high priority test cases on all browser and OS combinations if possible, then you can run your medium and low priority tests on a reduced list of combinations, that way, if you run out of time you’ll at least have tested the site for the largest segment of your audience and for the riskiest combination of browsers.

A key web testing rule is to remember that Javascript and Ajax will be handled by the web browser, which makes them likely to functionally differ between browsers but anything handled by the web server should behave in the same way on all browsers although you still need to test for styling issues. So if the new feature is a server calculated pricing component you only need to fully test the calculating logic once but you’ll need to test the price input and output fields on each browser/OS combination that is important to you.

 

So, in summary you should not expect to test every browser and operating system combination but with a little effort you can work out which ones are important or high risk for your project and then focus the majority of your test effort there.