Vision & Data

“Science without religion is lame, religion without science is blind.”
– Albert Einstein
While I won’t get into a debate about the merits of this particular quote, it did spark a discussion with a friend of mine about how various companies approach making whatever they make. Instead of the dependence between science and religion that Einstein talks about, instead let’s talk about vision & data.

With our newest project, we’ve released it in a relatively early stage. We push out new features as fast as possible, and we get a lot of feedback from our players. They both write us via our feedback form, and let us know what they think of the game, and they provide us data about what they’re actually doing in-game by simply playing, and letting us see the resulting data.

This is really, really different than how I’m used to designing a game, where there is a lot more distance between the designers and the players. Most of the time, you’d need to build a system, get it completed to the point where it’s very nearly finished, and only then do you get any sort of feedback from the end user. So it requires a designer to say, “Here’s what I think will work,” with relatively little supporting evidence other than that person’s vision.

There are a lot of companies that are embracing the idea of data-driven design. In this case, basically, we’re talking about a situation where you get a couple of alternatives, test them against some quantifiable measure, and go with the one that gets the best results. There are a lot of reasons this is incredibly successful – mostly because you only end up with things that you know are successful (at least in the way that you measured them).

If you look at this new crop of spam engines disguised as games, they’re highly data-driven – which is why, I think, people are content making the same games over and over – because these things have been proven to be successful – after all, they’re not measuring for creativity or originality. If you take a company like Zynga, who basically just cloned existing farming games, what they added was a highly data-driven approach to cranking up spam, and getting people to spend money.

Is that a bad thing? Well, it’s pretty mercenary, but it’s also the same thing that the folks at Bungie use to make sure you don’t get lost while playing Halo. It depends on who’s doing the data collection, and to what end. But that, gets back to the issue of vision. Why are you making what you’re making?

But at the same time, most game design is done almost completely blind. While it seems like doing a lot of A-B testing or focus testing should be so commonplace that no one even gives it a second thought, the level of data collection that’s done on *most* console games is still shockingly sparse. So you rely entirely on peoples’ guesses as to what the audience wants, not what the audience actually wants. And as a result, sometimes you have very talented people working in the completely wrong direction, and the end result is something that is in some way horrifically broken, not what people expected, or what they wanted.

I spent a little while trying to find some elegant way of rephrasing Einstein’s quote – but in this case, it really comes down to the fact that data without vision gives you something that lacks direction, while vision without data is just an incredibly risky endeavor. Not really a snappy rephrase, but whatever.

In the end, I think the only sensible thing to say is that you really need to balance the two. Game development is like a benevolent dictatorship – you want someone, or a small team with a strong vision for what they’re actually building that is listening to, and cares, about the audience. For us, the data-collection aspect of things is pretty new to us – we’re getting our sea-legs in that regard. The vision thing? Well.

You’ll have to let us know.