Does your application or API stand up to your users expectations?
There’s no doubt that customer or user satisfaction is high up on the list of things that are key to any application’s success. There are obviously lots of elements that contribute to satisfied users such as ease of use, the user interface, reliability etc. and of course we must not forget performance of the application, e.g. how long does it for the application to load or complete a transaction. Nowadays, customers and users expect instant results and rapidly become impatient with systems that take too long to load, in fact their threshold is around 2 seconds. Anything that takes longer than this to load will cause some users to look elsewhere”. It’s not just applications however, with an API, such as a RESTful service, performance is one element that contributes to user satisfaction or dissatisfaction and needs to be monitored and where necessary improved upon.
So just how do you begin to understand the responsiveness of applications or API calls and the impact that it has on user satisfaction?
Introducing the Application Performance Index (APDEX)
Let’s be clear, APDEX is not about timing things, it’s about “scoring”, similar to a Net Promoter Score it attempts to present how satisfied a user will be with the performance of that part of a system. Once you are measuring round trip page loads of your application or time to complete an API call you can begin to apply an APDEX score to it. APDEX (Application Performance Index) is an open standard owned by the Apdex Alliance (www.apdex.org
) and put simply is a way of summarizing measurements that can then
be interpreted on a scale ranging from excellent to completely unacceptable. What it doesn’t do is measure the round trip of a page load or an API call, that’s something you will need to provide to APDEX.
APDEX gives you a measurement of between 0.0 and 1.0 with 0.0 indicating all users are dissatisfied and 1.0 (a perfect score) being all users are satisfied based on the measurements that you supply it.
So just what is an APDEX score and how does it work?
APDEX is actually a very simple formula that aggregates and summarizes information based upon a set threshold, measuring the ratio of satisfactory to unsatisfactory results. For the rest of this article we will assume that we are measuring a page’s load time, or our response time. It is down to you to define or set the threshold (T) and all response times handled in less time than ‘T’ satisfy the user. So if you define ‘T’ as 2 seconds for a response from the application, and the response completes in 1.5 seconds then
you have a satisfied user and if above the threshold you would have a dissatisfied user. An APDEX measurement is taken using three response counts, Satisfied – The response is less than or equal to ‘T’. Tolerating – the response time is greater than ‘T’ and less than ‘n’ times ‘T’, typically ‘n’ being 4 and lastly, Frustrated – The response time is greater than the tolerating time. A simple formula is then
applied to capture the APDEX score.
APDEX = Satisfied requests + (Tolerating Requests / 2)
Total number of requests
Let’s give you an example so you can see what I’ve been talking about. We have a page of our application that is called 10,000 times per day and we have a threshold ‘T’ of 2 seconds for it to load.
During each call of the page we measure its “load time”. Assuming 35% of these complete within the threshold (3500) and a further 40% are tolerated (4000) and the remainder falls into the frustrated bucket. Our APDEX score would look like this:-
0.55 (APDEX) = 3500 + (4000/2) / 10000
APDEX suggests a good benchmark score is above 0.85 so in this simple example, given that 55% of our transactions are acceptable, it’s actually a poor result, as 45% of our users are finding the system frustrating, we should be aiming for an APDEX score of 0.90 or above. Clearly we can modify the threshold to get a more favorable APDEX score but we would really be kidding ourselves and ultimately still have an unhappy user base! However, we must be able to vary the threshold so that we can facilitate continuous improvement of our services until we have a happy user base.
As organizations become increasingly focused on Dashboards and KPI’s to understand the performance element of user satisfaction, APDEX can assist in quantifying this. In complex applications with many service components comprising a SOA you may have many areas summarized with APDEX scores to assist in understanding a transaction, but from the users perspective it is always the total transaction that influences success and leaves a positive or negative impression with the user. APDEX isn’t perfect, and can be cheated, so it relies on objective and fair metrics being set and measured. If you want to learn more about APDEX and how you might be able to use it you can visit www.apdex.org .