TGO Budgeting Solution – What Defines Acceptable Performance
Posted by Alan on March 16, 2011
When we sat down and sketched out the TGO Budgting Solution project we came us with a list of what we call our Critical Success Factors. Critical Success Factors are targets that have to meet in order to consider the project a success. It is a pretty standard thing we do here at TGO Consulting for any type of project. The key is that for a Critical Success Factor to be meaningful it must be objective and it must be measurable. For instance, a proper Critical Success Factor for us would be something like ”the form must open in under 5 seconds from the time the user presses the button” as that statement can be measured in an objective manner. What is useless as a Critical Success Factor would be something like “we need a cool interface”. The problem with a statement like that is who is the arbiter of what is ”cool” and how do you measure “coolness”. For instance, ask a dozen people now what they think about Charlie Sheen right now and you will probably get a dozen different answers.
For the budgeting project one of the Critical Success Factors we identified was performance, specifically end-user performance. We felt this was important because no one likes to do budgeting and it is considered by many to be one of the most evil components of their job. And although you can force them to do it, you can never make them like it and anything that makes the process more tedious will kill your user acceptance.
In most budgeting solution you have to run some pretty complex queries in the background before you return data to the end user. You can literally be combing through tens of millions of rows of data. Now while using a cube architecture helps performance, not all budgeting process lend themselves to cubes. Any process where you need to add records on the fly, such as Compensation/HR and CapEx, tend to not work well in a cube world. And since our solution plans to support those types of models, we have to support relational. And anyone who has tried to access large data sets while pivoting on multiple columns within a SQL Server query will tell you — performance can easily go to the dogs.
What we are struggling with is just how long a wait is too long? Einstein proved that time is relative. Think of it this way. The hour you spend holding your wife’s purse while she tries on different dresses in the department store seems like an eternity against the hour spent in the bar watching the game while drinking beers and eating chicken wings with your buddies. Both were still an hour. In the budgeting world, when I first get a budget assignment, waiting for the form to open or for my data to save may be acceptable. However, when budgets are due in 5 minutes and I am rushing to complete before my deadline then even a minimal delay will drive a user insane.
So what should we be using as a target for how long a budget entry user has to wait to refresh and/or save their data? 2 seconds is perfectly acceptable (in my mind anyway) and 1 minute is definitely too long. So given instant is ideal and 1 minute is like going dress shopping what is the maximum time users will find acceptable? Is 5 seconds too long? How about 10 seconds? 15? 20? 30? Where do we have to draw our line in the sand?
I would love to get thoughts from any of you on this.