Attendees at the O'Reilly Velocity conference in June were treated to the unusual phenomenon of a joint presentation by Google and Microsoft. The presentation outlined the results of studies by the two companies on the effects of search response time (see http://bit.ly/pkV4x for the full presentation). Aside from the novelty of Microsoft-Google cooperation, the presentation was notable both in terms of its conclusions and its methodology.
It's obvious that application performance is a key factor in user productivity. A slow application diminishes the productivity of the user of that application. In the online world, poor performance can also lead to users abandoning one application in favour of another: if Google is slow, perhaps I will use Yahoo.
Clearly, the search vendors have a pressing need to understand exactly how application performance affects their users. To this end, both the Microsoft Bing (www.bing.com) and the Google teams have been conducting experiments in search engine performance. These experiments have involved artificially manipulating the response time for a small proportion of users and then comparing those users' behaviors with a control group of unaffected users.
The results of the experiments are fascinating, and say as much about human psychology as computer technology.
The Bing group found that introduced delays of 200ms or more (1/5th of a second) resulted in user response delays of about twice that amount. For instance, if Bing introduced a one-second delay in the page load, there was an almost two-second delay created in a user's time to respond. When pages load slowly, the reduction in the focus and attention appears to add as much delay as the increase in computer response time.
Similar experiments at Google showed that a user's average number of search requests diminished as the response time delay increased-and that the effect got worse over time. Each week that a user experienced delayed response time decreased the average number of Google searches per day. This might be because the user defected to another search engine, or declined to perform a search because it was perceived as too time-consuming.
When Google removed the response time delay, user search rates increased almost immediately, but did not instantly return to the baseline level. Users don't immediately forget or forgive poor performance.
Bing found that the effect of response time delays were reduced if at least part of the page was rendered quickly; this probably reduces the subjective response time and helps maintain user attention and focus.
These results have clear implications for application architects. The cost of poor response time is greater than the raw computer wait time-it also decreases user productivity through reduced attention and focus. The longer an application provides poor performance, the greater the effect.
When you can't reduce the objective response time, decreasing the subjective response time can help. Providing quick rendering of at least part of the results, as in the Bing study, is one way to do this. Other studies have shown that progress bars and other feedback mechanisms also decrease subjective response time.
What application developers think of narrowly as application response time is actually only one half of the equation. The real measure of application productivity includes both application service time and human response time. Maximizing the human response time involves both psychology and computer science.