Alan Whitehouse's Ramblings

Continuing to work until my heavy investment in lottery tickets finally pays off….

  • Categories

  • Archives

  • Deep Thought

    Historically speaking, all true change in the world has come thanks to leaders emerging, them taking charge and giving the masses someone to rally around. Can an intentionally "leaderless" movement survive or will it just slowly fade away?

  • Subscribe

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 19 other followers

  • Me!

  • Disclaimer

    The views expressed in this blog, while intelligent and always right, are strictly my own, and do not necessarily reflect the views of anyone else with which I am in any way affiliated. And don't forget, I own the rights to all information on this blog (except for the stuff I stole from other people).
  • Current Rant

    *START OF RANT*

    You want change, then get involved. Vote, run for office, go to shareholder meetings and contact advertisers or investors. Sitting around banging drums, singing kumbaya, smoking weed and having a camp out under the stars is not going to get you the change you want.

    *END OF RANT*
  • Admin Stuff

TGO Budgeting Solution – What Defines Acceptable Performance

Posted by Alan on March 16, 2011

When we sat down and sketched out the TGO Budgting Solution project we came us with a list of what we call our Critical Success Factors.  Critical Success Factors are targets that have to meet in order to consider the project a success.  It is a pretty standard thing we do here at TGO Consulting for any type of project.  The key is that for a Critical Success Factor to be meaningful it must be objective and it must be measurable.  For instance, a proper Critical Success Factor for us would be something like “the form must open in under 5 seconds from the time the user presses the button” as that statement can be measured in an objective manner.  What is useless as a Critical Success Factor would be something like “we need a cool interface”.  The problem with a statement like that is who is the arbiter of what is “cool” and how do you measure “coolness”.   For instance, ask a dozen people now what they think about Charlie Sheen right now and you will probably get a dozen different answers.

For the budgeting project one of the Critical Success Factors we  identified was performance, specifically end-user performance.  We felt this was important because no one likes to do budgeting and it is considered by many to be one of the most evil components of their job.  And although you can force them to do it, you can never make them like it and anything that makes the process more tedious will kill your user acceptance.

In most budgeting solution you have to run some pretty complex queries in the background before you return data to the end user.  You can literally be combing through tens of millions of rows of data.  Now while using a cube architecture helps performance, not all budgeting process lend themselves to cubes.  Any process where you need to add records on the fly, such as Compensation/HR and CapEx, tend to not work well in a cube world.  And since our solution plans to support those types of models, we have to support relational.  And anyone who has tried to access large data sets while pivoting on multiple columns within a SQL Server query will tell you — performance can easily go to the dogs.

What we are struggling with is just how long a wait is too long?  Einstein proved that time is relative.  Think of it this way.  The hour you spend holding your wife’s purse while she tries on different dresses in the department store seems like an eternity against the hour spent in the bar watching the game while drinking beers and eating chicken wings with your buddies.  Both were still an hour.  In the budgeting world, when I first get a budget assignment, waiting for the form to open or for my data to save may be acceptable.  However, when budgets are due in 5 minutes and I am rushing to complete before my deadline then even a minimal delay will drive a user insane.

So what should we be using as a target for how long a budget entry user has to wait to refresh and/or save their data?  2 seconds is perfectly acceptable (in my mind anyway) and 1 minute is definitely too long.  So given instant is ideal and 1 minute is like going dress shopping what is the maximum time users will find acceptable?  Is 5 seconds too long?  How about 10 seconds?  15? 20?  30?  Where do we have to draw our line in the sand?

I would love to get thoughts from any of you on this.

About these ads

4 Responses to “TGO Budgeting Solution – What Defines Acceptable Performance”

  1. Peter Eb. said

    A hard limit of 5s is definitely hard for the coder. Depending on what you need to do when you click the button I like to look at the incremental cost. If you are opening an xl workbook from sharepoint that might take 5s alone in the worst case. So how much longer does it take to open the budgeting bits? 50% 100% of the workbook open time? Or do you want to make it 5s more than the workbook opening alone. I prefer always using the multiple from the baseline scenario, so that when somebody tries to use this over dialup you don’t fall a foul of any perf guarantees that might be negotiated into later contracts. (because you’ve educated your internal team that you are not magic and they can have rational conversations with customers about how this is doing more than opening a workbook etc etc)

    On the other hand, when you get customers asking why XL used to start in less than 1s on their machine, and after installing your addin on their machine it takes 2s and they are complaining of over 100% slowdown… its another kind of difficult situation (And at the time, on my dev machines I had never seen XL start in less than 3s!) Definitely its all relative

  2. Alan said

    Thank Peter. From my point of view, at this point we are ruling out the actual opening of Excel as part of our performance tuning as that is totally out of our control. For instance, I had to rebuild my laptop from scratch a couple days ago and when I first launched Excel it was blinding fast. I then intalled Power Pivot and the base opening of Excel has slowed down. So my times are against 3 general buckets — 1) opening a budget template after Excel is already open, 2)refreshing data after a user makes their dimension selections and 3) saving data back to the database.

  3. Peter Eb. said

    So even for the relational query kinds of things, I’d pick the simplest query you can run, and think about how to relatively benchmark off of that as well. If you can repro that in a customer environment too that will help troubleshoot. Of course convincing the customer that its not your fault its slow is left as an exercise for the reader.

  4. Andrew S said

    As long as you don’t use Perform in the name of your budgeting tool, I think performance is something that is really relative to the environment and the benefits vs a manual or semi-automated process. You could trade off performance with features and mask with distraction. To speed things up appearance-wise for slow database calls, you could use asynchronous calls to improve the user experience.

    Would also suggest that you can always tweak performance of pivoting SQL with summary tables, numbers table, or use CLR for complex logic to improve things further, though many times you’re at the mercy of your client’s SQL install performance. Sometimes tasks like pivoting and transforms are better done outside the data layer…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
Follow

Get every new post delivered to your Inbox.

%d bloggers like this: