Recording and Understanding Mobile People and Mobile Technology by Paul Tennent and Matthew Chalmers, in proceedings of the first international conference on e-social science, Manchester, UK.
We present an approach to recording and understanding the activity of people moving and interacting with each other via technologies such as mobile phones and handheld computers. Our focus is the combination of observational techniques, usually based on video recordings, and system–based techniques that log or instrument the technologies in use. At a higher level, we explore tools to allow sociologists and computer scientists to interact around a coherent visualisation that coupled resources usually associated with just one of these two communities of research practice. The Replayer system supports the creation of system logs, and the visualisation of the results in a display that is synchronised with video and audio recordings. We present a case study showing how Replayer was used in the evaluation of a mobile multi-user system called Treasure, highlighting evaluation results that would not easily have been discovered by more traditional means.
Why do I blog this? because with CatchBob! we are also working on this issue; we have a replay tool that we use to analyse the data extracted from the game and confront it to the players. They then have a trace of the activity on which they can reflect to explain us what happened. Paul Tennent has an interesting webpage in which he describes more this topic, which is his PhD resreach issue. I like his conclusion:
Location-aware mobile systems are by their very nature difficult to evaluate. Combining logs from multiple sources to create a coherent view of a system state, can prove to be of great value when performing post-hoc analysis of a system however this usually requires considerable effort and often custom built tools. This system aims to simplify this process by providing a generic toolset for the creation and visualisation of log data, applicable across a wide variety of applications.