Lies Your Designer Told You (or Data vs Design)

March 27, 2015

Designers versus data more than ever deserves its place in the pantheon of great conflicts: the Hatfields vs. McCoys, Android vs. iOS, Social Media Marketing vs. Results, Athens vs. Sparta, the Doctor vs. Daleks, Auburn vs. Alabama, and Fox News vs. reality.

We make this out to be some great collision of disciplines when in fact they are not opposites and they can and should work together.

To do so requires the dropping of a lot of false constraints that we place them in. Its all the lies that we tell others and especially the ones we tell ourselves that stop us from doing the very simple tasks needed to really drive performance. Figure out the lies and figure out what really matters and so much noise simply floats away.

What is data and what is design?

Data is simply a rational way to make rational decisions which are best for the business. Design is about creating different useful experiences, so in concept they should be best friends and work together to simply do what is best for the business. So why then do they constantly fail to do so?

Why are so many designers adamantly opposed to testing and data in a rational sense? Why does so much time get wasted on heuristics, storytelling, and well – nonsense?

We’re all wrong

The truth is that both sides of the debate are guilty of many sins. Analysts often confuse using data to further their opinion for actually improving business. “Creatives” do not want anyone to interrupt their vision of the way things should work. Both sides often fail to stand up to voices from up high and both are often swayed to change their work to match the current prevailing winds, be it from executives or from the industry as a whole.

Both sides tend to be extremely mentally lazy and rely on what feels good or makes others happy instead of diving deeper and forcing rational decision making. Because of this I want to investigate many of the common “lies”, things that are said to defend a point or to push an agenda that have no basis in a rational decision making world, in order to show just how much of the common talk and knowledge about user experiences is just complete BS.

The Many Lies of Design

Here we go.

It’s About Good User Experience

No phrase has less meaning and yet is used to defend so much as doing something in the name of a good user experience. Just for once, I would love to get 5 people in a room and have them individually define this concept with detail and get them to all agree. It is just a cotton candy saying, one full of air and that rots your teeth, and yet no phrase acts as a shield quite like it.

I am not here to say build an awful experience, I am saying that using empty jargon that you feel justifies opinion is a waste of everyone’s time. What is a good user experience? My answer is always one that allows the most people to do the thing that is best for them and for the business. It isn’t one that gets them to look at a specific thing or gets them to look at an interesting picture, it is the one that accomplishes the most for both parties.

This goes back to why a single success metric is so important. It allows you to frame the discussion away from empty jargon or what people think will happen and instead focus solely on the real purpose of the site and of the action you are taking, to generate more revenue. If the “good user experience” that they are arguing about is better for the user, then more of them will do what they need to do to make your site money. If it isn’t, then empty debating just wastes time and finding that out generates more revenue.

This doesn’t mean go out of your way to piss off your users, but it does mean that you need to challenge every part of what you think makes a user experience “good”. The more rules you put in place, the more you are artificially limiting the results that your optimization and organization can drive.

We Need to Provide a Consistent User Experience

From an optimization perspective, this one comes up the most often. You are by definition changing small or large parts of the whole, and the fear and that this will create an inconsistent user experience gets brought up all the time. You need to have a consistent look and feel to your user’s journey… why is this a thing? Why do you care? This is the definition of a rule for the sake of a rule.

Sometimes a consistent user experience matters. Sometimes it is counter-productive. There is nothing easier than testing to figure out the answer. In almost all cases you will be testing the “inconsistent” experience, so if they are really worried about the impact, it is the worst case scenario that you are testing, not the best.

If you find something you can test and see if the same concept applied elsewhere does improve outcomes or is just a waste of time. Arguing about it before the fact is the same as having a heated debate over whose turn it is to clean the bathroom when the entire house is being flooded. It is a debate, but it is very low on the things that matter and you should probably worry more about fixing the leak instead of who gets to do what later on.

You Should/Shouldn’t Copy Competitors Designs

I think by now all have heard the idiom, “Good artists copy; great artists steal.” Too much time gets caught up on focusing on your competitors designs and websites, leading to many to have a visceral reaction to the concept of using parts or all of their designs. The truth is that it is not the copying that is the problem, its the competitors part that is where all the failure takes place.

Why does it matter if a design is your competitors or if it is a completely different industry? The only thing that should matter is getting results and figuring out where to apply resources. This means that you need to look everywhere for things that can add to a test by increasing the beta of your options. Look at Google, look at Amazon, look at small lead gen start-ups and eCommerce and ticket platforms and NPOs and everywhere to find concepts that you can leverage or copy directly to including in a range of outcomes.

In the end it doesn’t matter if an idea is new or a direct copy, it doesn’t matter if your arch rival or a company you never heard of came up with what ended up winning. In fact what ended up winning DOES NOT MATTER at all, the only thing that matters is managing the input and acting on the data to maximize outcomes.

This reminds me of a story of when I was consulting. At one point I was working with 5 of the largest travel sites and airlines at the same time. Without knowing it, all 5 wanted to focus solely on the quick selector item on their front doors, what most called their “cannonball”. At some point all 5 of them brought up at least 2 of the others, and months were spent focusing on that one item.

Once they were done, I was able to get them to test other places and to focus far more on discovery instead of pointless internal debate and you know what happened? Not a single one of the sites ended up with a design like the other and not 1 had the front door – let alone that part of the front door – end up being the most influential page on their site. So much time wasted, and so much energy spent debating what their competitors were doing, all for nominal gains.

You Need to Speak to Your User

I for one am all for removing the need to have your users hear voices in their head. That is exactly what I think of every time someone throws this jewel out – be it a designer, a copywriter, or anyone else. You need to create the most efficient and valuable experience, period. If it involves lots of text or creating a dialogue – great. If it involves a single entry bar and two buttons, then also great.

You know what you should do? Test both and other points in between and let the data tell you. And when you get a larger design that works, then you should test to figure out what influence your single behavior and then optimize that.

I would also add the second meaning of that phrase which is how much you put value into user feedback and qualitative data. While it may inform or give you 1 or 2 new variants that you would not have gone with before, the question is how much you can trust their feedback (perhaps not at all) and what the cost/benefit ration is of that exploration (hint: It is awful). You need to generate as many different and high quality ideas as possible, and so if you need to go that route it is not the end of the world, but it is hands down one of the most misleading and one of the least efficient ways to generate variants.

Good Design is Long-Lasting

Not to throw Dieter Rams under the bus, but why is it long lasting? Why does it matter that we come up with something that will be a static thing that we need to stay with for years?

You have to think of every part of the experience as an ever changing amorphous design that allows your users to change and your site to adapt to them. Getting caught up on what worked in the past, or trying to create something that will work 10 years from now is completely pointless. It matters in what it accomplishes, how much cost it was to create it, and how much you can exploit it. This does mean that the longer you can exploit something does weigh in, but not as much as how fast you can adapt and how large the beta is in the concepts that you test.

This is also why adaptive decisioning agents are going to continue to become more and more powerful. Creating a system that automatically can adapt and change based on the current data, and feeding that system with a large array of quality options helps maximize the long term performance while minimizing the value and consistency of any one idea. The larger the array of concepts and the less rules you put on that system, then the better the outcome.

Make a Design for Each [Persona/Market Segment/Target/Geography]

Personas and personalization, the bain of my existence. If it helps you come up with a different design that is part of a larger series of testing, then great and you should allow that type of mental model and concept creation. What makes it irrational is when you get stuck on that concept, or when you refuse to discover if your concept works like you think or just maybe works for an entirely different segment of users.

Should you have multiple experiences? Probably, but not always and only when it is far more efficient to do so. What should the division be? I have no clue and I guarantee you do not either. It may be the persona you spent hours building (just kidding, it won’t be) or it may be based on geography, or referrer, or mobile, or time of day, or browser, or user behavior or the weather or any other way of dividing users.

You know what the beauty of testing is? You get to serve all experiences to everyone and then let the data tell you where to go. You can’t lay your hands on what goes where or to whom and you most definitely can not limit yourself to just one type of experience in a rational and high performing environment.

A great goal for a site should be 4 defined user experiences on their site after a year.

This means two rules that change the user experience based on actual results that proved it was worthwhile and consistent to do so. It is hard to find large meaningful exploitable experiences, implement them, maintain them, and then continue to optimize them. They are extremely valuable when done right, but in all other cases it is just a waste of time and energy and almost always results in lower results than just doing nothing.

Good Design Looks Good

It has become a running joke at my current and last job that whatever people like the most or think looks the most appealing is doomed to complete failure during the test. Time and time again the ugly stepchild option wins, yet there are still people caught up on how much they like the visuals of a specific design. Even worse when the one they like actually does win you get comments like “of course it won, it was the only good looking one.”

Good design is about its ability to do work. It is not art, it doesn’t need to create an emotional response, it just needs to do its job. That means that you can’t look at something and know that it is going to work or not with any certainty. The very act of looking at something and making a judgement is subjective, whereas comparative measured data evaluation should be rational. While you can pick up some patterns about things that have a slightly higher chance to work, the reality is if you cut off what you test or what can possibly test then you are by definition limiting the possible outcomes of a test.

You don’t have to like what wins, you just have to like getting results.

What is the truth about Design and Data?

The simple truth is that data is the best thing that has ever happened for designers. Where before they were stuck with fighting for a vision, for doing everything in their power to push their concepts past reviews and executive feedback.

Where they had to design for what impresses people, now they can free themselves from the shackles that have plagued their discipline for so long.

Designers may not like it, it will definitely be uncomfortable, but if they can embrace it then they can stop worrying about vision and feedback and all of the lies mentioned above, as well as just about every other rule of “good design”.

In many ways the real job of the optimizer is to disprove everyone of these rules as often as possible. In order to do that they have to constantly creating a system that enables many inputs and influencing the range of inputs to maximize the chance for that very thing to happen. Optimization is about building a rational decision system and then not allowing any person or bias to knock it off its tracks.

The key of a great designer is to come up with as many different feasible high quality designs and then hand them off to the users. They don’t have to understand all the users or especially what wins, they just have to be open to many different ways of tackling a problem or even about what problems really matter. The better they can feed a system with quality and quantity of ideas and the less they get stuck on what comes out of the system, the better for all concerned and especially their organization.

Conclusion

One of my favorite descriptions about the difference between art and design is that design is art that does work. As an optimizer and as someone trying to enforce data rationality, the key is that I do not care about the art, I care about measuring just how much work it does. It is not good enough that it does work, it is about picking what does the most work. To do that both sides have to be open to going wherever that data tells them and to go against just about everything that is wired into their brains.

Opinions are nice, results are better.

Digital & Social Articles on Business 2 Community