User research and informed opinions

An interesting sidebar from an old issue of game developer (november 2007) called "usability research commandment" by Randy Pagulayan (Microsoft Game Studio user reasearch) deal with the relationship between user experience researchers and designers. Some excerpt that I find interesting and relevant beyond the game field:

"Be flexible, it is our job to try an account for as many sources of bias and influence when we run usability tests and collect data, but sometimes the ideal is simply not practical. (...) Users have opinions, but designers make the call. During your research and testing, users will always have opinions on things they do or don't like. Your job isn't to adhere to user whims - your job is to identify areas where user behavior is not consistent with the design's vision. What you do from there will be context dependent. (...) Most developers aren't interested in the classic "it depends" answer to something [very academic]. They also aren't interested in inferential statistics, hypothesis testing, or the number of users you need for a valid test. When asked to do something or answer a question, do your research and testing, and give it your best shot. Don't be afraid to have an informed opinion, even if your research wasn't suitable for a scientific peer-reviewed journal."

Why do I blog this? All of this rings a bell with my current practice. There's even more to be quoted here but it's certainly that last bit which caught my attention. Working with designers for a while, I certainly shared that sort of feeling about what sort of material I needed to bring to the table to help them. However, it does not mean that the result should be overstated. As Pagulayan says, "What you do from there will be context dependent". Also see how Jan says about this notion of informed opinion and the risk of overstating:

"So why should anyone give your research the time of day? How to build credibility? For starters recognise and communicate the limits of (mostly qualitative) design research. We start out with opinions, and all things by the end of study we move onto having informed opinions or on rare occasions very informed opinions. Overstating the value of the research makes you a bullshitter."