An interesting CSCW workshop about colocated collaboration evaluation was set during CSCW 2004 in Chicago. The paper are avalailable here. I hope they will put the folllow up as well.
One of the major challenges researchers in this area face is the difficulty of assessing the impact that the technology has on collaborative behaviour. For example, what measures can we use to understand whether the technology has enhanced or compromised the collaboration? Many researchers make methodological choices based on a specific configuration of an environment, or based on what resources are currently available to them. As such, many of the methods lack the reliability or robustness that would come from established, more general approaches, which could be better validated if applied across a variety of experimentations. In addition, tasks chosen to evaluate co-located collaborative environments are often selected in an ad-hoc manner, dependent on the interests of the researchers or on the appropriateness for a specific environment.
This lack of established guidelines for evaluation of co-located collaboration means that it is hard to calibrate many of the research contributions or truly understand the impact of the technology. This workshop proposes four themes related to evaluation of co-located collaboration behaviour:
- Impact of technology on group interactions
- Impact of technology on social dynamics
- Impact of individual personalities and interpersonal dynamics
- Choice of appropriate tasks