[Research] Field Experiments: between field study and lab experiment
I really appreciate the concept of "field experiment" described by Goodman, Brewster and Gray (2004) in their paper Using Field Experiments to Evaluate Mobile Guides.
Field experiments are quantitative experimental evaluations that are carried out in the field, drawing from aspects of both qualitative field studies and lab experiments. To start with, field experiments have advantages over expert evaluations (such as heuristic evaluations and checklists) because they involve real users. This is particularly important when studying novel, variable and less understood situations, such as those involved in mobile devices. Secondly, there are various reasons for preferring field situations to laboratory settings. Principally, the difference between use in a laboratory setting, even when mobile conditions are simulated, and use in the real world can be quite startling. This is particularly true for devices such as mobile guides that rely heavily on the surrounding environment. Aspects such as lighting levels, weather, the effects of walking, the appearance of landmarks in real life and the effectiveness of location-sensing systems can have unpredictable effects on the usability and effectiveness of a device. The only way to really see how the device will work in practice is to use it in practice. In addition, field experiments have various advantages over more qualitative and ethnographic field studies. Although such field studies do have some advantages, it can be hard to use them to obtain an objective evaluation of a device, determine its performance or gain hard evidence comparing one device or method with another. Field experiments offer one way to overcome these disadvantages while still utilizing the advantages of a field setting. They are also quicker than carrying out a fullblown ethnographic field study and can gauge the responses of a greater number of participants.
It's closely related to the notion of "quasi-experiment" developed by T.D. Cook, D.T. Campbell, Quasi-Experimentation: Design and Analysis Issues in Field Settings, Rand McNally, Chicago, 1979.