Clumsy automation and user surprises
One of the outcome of my research lately is that automating location-awareness might be detrimental to group collaboration in mobile settings (more about it in this paper .pdf). The concept of automation drawbacks has been already adresses in human-computer interaction and is often refered to as "clumsy automation". This is developed in David Woods' paper called "Human-Centered Software Agents: Lessons from Clumsy Automation". Some excerpts:
These machine agents often are called automation, and they were built in part in the hope that they would improve human performance by off loading work, freeing up attention, hiding complexity -- the same kinds of justifications touted for the benefits of software agents. (...)
The pattern that emerged is that strong but silent and difficult to direct machine agents create new operational complexities. In these studies we interacted with many different operational people and organizations, * through their descriptions of incidents where automated systems behaved in surprising ways, * through their behavior in incidents that occurred on the job, * through their cognitive activities as analyzed in simulator studies that examined the coordination between practitioner and automated systems in specific task contexts, * unfortunately, through the analysis of accidents where people misunderstood what their automated partners were doing until disaster struck.
In our case, the automation did not create cognitive or physical workload nor incidents or suprising things BUT it lead users to a certain inertia in terms of communication (they communicated less), strategy planning (they did not reshape their strategy). Users considered that the information given by the automatic location-awareness tool was sufficient to complete the task and that's it.
Ref: Woods, D. D. (1997). Human-centered software agents: Lessons from clumsy automation. In J. Flanagan, T. Huang, P. Jones, & S. Kasif, S. (Eds.), Human centered systems: Information, interactivity, and intelligence (pp. 288--293). Washington, DC: National Science Foundation.