This week in uxclinic: Death by comparison
This week in the ux-clinic discussion forum – Death by comparison:
I’m a usability engineer on a major web site. Our senior managers are addicted to data. Hard core data. They make all decisions based on metrics and what the call “the metric function” – the equation that best determines what success is.
So when it comes to usability, the only studies they’re interested in are comparative ones, where I do A/B testing, or in some cases A/B/C testing. Even when we prototype or experiment, they always want the data housed in comparative data.
How do I get them out of this data rut and recognize that usability engineering involves more than generating numbers to put in charts? Or is this how most of the tech sector sees usability: a number factory?
I have a highly sarcastic mind.
So I would play with metrics and force them to make a decision based on logic and not numbers.
Like give them a tied score and say – “The metrics say that either A or B will do. Now you decide which one you want?” If they still insist on more data, generate more of the fudged statistics.
It may sound unethical, but eventually when you know that certain processes are not completely metrics-driven (as opposed to information-driven or logic-driven) then it makes pure sense to follow the right path.
This is exactly like the project manager who is shown code when he demands it, even if the project is in the inception phase. Sometimes ignorance is much more than bliss.
There’s a great commercial out right now that addresses this. It goes from one company conference room to another and each say, “I wonder how XYZ company would do this?” The conclusion being that no one will be able to stand out if they all just follow each other’s lead.
Web Design is relatively unique in that your competitors designs are largely available to all and can be scrutinized at will. This certainly breeds the comparative approach your manager’s subscribe to.
Of course, the stated goals of your company are important here. In some industries it does not pay to be different. Making a mistake is more costly than the benefit of being a hit. In other industries the success of your web site could make or break the company.
If your website is not a mission critical part of your business, a safe comparison-based approach is probably sensible. I’m not a fan of sameness myself (I find it boring), but it has advantages. Visitors to your site will at least find a familiar (if not innovative) environment and the cost of creating and maintaining your site should be lower since you’re following the beaten path.
Maybe a career change into the Arts is in order? Good luck.