IEEE on interactive sonification

in

IEEE Multimedia's last issue is about interactive (with) sonification. The point is that sonification "presents information by using sound (particularly non speech), so thatthe user of an auditory display obtains a deeper understanding of the data or processes under investigation by listening". Interactive sonification is then defined as "the use of sound within a tightly closed human–computer interface where the auditory signal provides information about data under analysis, or about the interaction itself, which is useful for refining the activity".This issue provides the reader with a good review of cutting edge projects:

  • Zhao et al. report on “Interactive Sonification of Choropleth Maps.” The extension of visual maps is not only interesting for blind people, it also inspires us to consider the extension of other visual techniques into the auditory domain.
  • Fernström, Brazil, and Bannon present in their article, “HCI Design and Interactive Sonification for Fingers and Ears,” an investigation of an audio-haptic interface for ubiquitous computing. This highlights how human beings can use the synergies between data presented in different modalities (touch, sound, and visual displays).
  • In their article, “Sonification of User Feedback through Granular Synthesis,” Williamson and Murray-Smith report on the progress in the domain of high-dimensional data distributions, one of the most appropriate applications of sonification.
  • From a completely different angle, Effenberg discusses in his article, “Movement Sonification: Effects on Perception and Action,” the enhanced motor perception in sports by using an auditory display. Effects on perception and action are reported from a psychophysical study.
  • In “Continuous Sonic Feedback from a Rolling Ball,” Rath and Rocchesso demonstrate the use of an interface bar called the Ballancer. Although this interface is not yet used to explore independent data, it is an ideal platform for studying the interaction at the heart of an auditory interaction loop.
  • Hinterberger and Baier present the Poser system in “Parametric Orchestral Sonification of EEG in Real Time.” The electroencephalogram is an interesting type of signal for sonification because it involves temporal, spectral, and spatial organization of the data.
  • Finally, in “Navigation with Auditory Cues in a Virtual Environment,” Lokki and Gröhn show how sonification can enhance navigation and operation in spaces that so far have only been explored visually.

My favorite is Fernström, Brazil, and Bannon's project.

Raytheon, Minority Report and gesture-based technology

in

An interesting story in the Wall Street Journal: 'Minority Report' Inspires Technology Aimed at Military.

In the futuristic movie "Minority Report," Tom Cruise gestures with his gloved hands to sift through crime-clue data that are displayed on giant screens. With the twist of a wrist he can move information from one column to another or delete items. (...) Raytheon then hunted down the scientist who was behind the movie technology, John Underkoffler. Raytheon decided to fund an effort to try to turn his film fantasy into reality and explore its potential for speeding up intelligence analysis (...) The fruits of that investment are housed in a darkened room in a converted Los Angeles factory. There, a man wearing reflective gloves uses hand gestures to manipulate pictures projected on a panoramic screen. He slides an index finger forward to zoom in on a street scene; swivels a horizontal hand to the right to scroll through a video; sweeps both hands to the left to clear the screen. Raytheon believes such "gesture technology" can help solve one of the military's biggest problems: information overload (...) Raytheon isn't alone in chasing the command post of the future. And it isn't the only company injecting Hollywood into this race. Silicon Graphics Inc., which is known for special effects in movies, is working with the Army to develop the computing firepower that command centers will need.

And on a different note, video games are also looking in that direction:

Raytheon is working on more immediate applications, such as a device called a Common Tactical Blackboard to offer a portable bird's-eye view of a battle zone and software that suggests combat responses. But Mr. Underkoffler retains the right to pursue commercial uses, such as command-and-control operations for railroads and ports, and virtual wind tunnels for industrial designers. Videogames are also in the mix. With similar but less advanced technology, Sony Corp. already markets the EyeToy, in which a camera captures a person's movements and incorporates them into the game on the TV.

Paper in Psychnology about socio-cognitive functions of space

in

My literature review about the socio-cognitive functions of space has been accepted for a special issue of Psychnology about Space, Place and Technology. The paper is entitled "A Review of How Space Affords Socio-Cognitive Processes during Collaboration". Here is the abstract:

This paper reviews the literature about social and cognitive functions of spatial features used when collaborating in both physical and virtual settings. Those concepts come from various fields like social, cognitive as well as environmental psychology or CSCW (Computer Supported Collaborative Work). We briefly summarize the social and cognitive affordances of spatial features like distance, proxemics, co-presence, visibility or activity in the context of physical and virtual space. This review aims at grounding in an explicit framework the way human beings use space to support social interactions. This review can be used as a starting point to design efficient applications that take spatial context into account.

It will be online in few weeks I guess.

Poster for HCI 2005

in

I'll be at HCI 2005 In Las Vegas to present a poster abou my PhD thesis. The short paper might be downloaded here. It's mostly the rough description of my first experiments' results.

A Mobile Game to Explore the Use of Location Awareness on Collaboration by Nicolas Nova, Fabien Girardin and Pierre Dillenbourg
This contribution presents an ongoing study focused on how location awareness feature modifies collaboration in the context of mobile computing. First it describes the environment we designed and implemented in the form of a mobile game called CatchBob!. This application running on TabletPCs engages groups of three participants in a collaborative treasure hunt over our campus. The game is used as a platform to run field experiments to get empirical results about how information concerning partners’ whereabouts impact collaborative processes. We are interested in processes such as division of labor, the inferences made by participants about others activities and the building of a shared understanding of the situation. Players can communicate by drawing information on the TabletPC that displays a campus map. Those drawings are broadcasted to each participant. Finding the object was achieved through a proximity sensor that indicates how close the user was from the virtual object. Collaboration among the peers lies in the fact that they had to surround the object with a triangle formed by their positions. We tested two experimental conditions. In one condition, users could see their partners’ positions. In the other condition, participants were not given location-awareness. This poster presents the game, how it enables us achieving our goals and specifies which kinds of data we are able to extract. We then report the results of a study we conducted. According to our ongoing experiment, there seems to be no differences between the two conditions with regard to the task performance. However, players without the location awareness indications have a better representation of their partners’ paths. This is due to the fact that they annotated more the shared map: positions indications (to compensate the absence of location-awareness) but also directions and strategy messages.

ipod to ipod connection

in

icopulate is a new tool for ipod fans:

Now your iPod® can get some action and do direct data transfers at the same time! Here's how it works. Simply apply a dab of the included iLube© onto the dock connector of your iPod® and slip it into one end of the iCopulate's™ latex sleeve. Find another consenting iPod® in the vicinity and deftly insert it into the opposite end of the sleeve.

Once the two iPods® are joined the real fun begins. Using the easy to navigate menu system of your iCopulate™, you can transfer music from one iPod® to the other; single tracks, albums, all songs, or even whole playlists. Since the iCopulate™ uses the dock connector for fast file transfers you can transfer several dozen tracks in less than 30 seconds. Now that's a quickie you can write home to mom about!

MMORPG addiction factors

in

Nick Yee has an interesting take on addiction factors used by game desginers in MMORPG. To put it shortly, there seems to be 3 main attraction factors of MMORPGs that encourage time investment and personal attachment:

  1. the elaborate rewards cycle inherent in MMORPGs that works like a carrot on a stick. Rewards are given very quickly in the beginning of the game. You kill a creature with 2-3 hits. You gain a level in 5-10 minutes. And you can gain crafting skill with very little failure. But the intervals between these rewards grow exponentially fairly quickly. Very soon, it takes 5 hours and then 20 hours of game time before you can gain a level. The game works by giving you instantaneous gratification upfront and leading you down a slippery slope. And it overlays different reward cycles so you're always close to some reward - whether this be a level, a crafting skill, or a quest.
  2. the network of relationships that a player accumulates over time. There are several reasons why relationships of a platonic or romantic nature occur so frequently in MMORPGs. The anonymity and computer-mediated chat environment facilitates self-disclosure, and many players have told personal issues or secrets to online friends that they have never told their real life friends or family. The high-stress situations inherent in the game also help build trust and bonds between players very rapidly. Of course, another important reason is that the games were designed so that you have to group to achieve most goals.
  3. the immersive nature of these virtual environments. This factor works by encouraging players to become attached to their characters and the virtually valuable items that they own. The immersive nature also encourages players to become personally invested to what happens to their characters, and to be empathetic towards their characters. In the same way that a movie or fairy-tale enchants you, the immersive quality of MMORPGs tries to enchant you with a fantasy, and make you feel that you are part of something grand and extraordinary.

He gathered these information through a research process detailed here in the context of his Ariadne project.

Playshop: an open-access laboratory to create botanical gameboys

in

Playshop is described as an "an open-access laboratory which encourages the free flow of ideas. It presents projects, workshops, seminars, art installations and a web site that collectively question or challenge the role of technology and propose alternatives to the cultural social and economic systems we live in. Playshop is where the energy of art production, education, curatorial practice and social interaction fuse to create a vital public space and an environment of exchange. "I like some of their projects:

The projects include: the Fingerprint Maze, a physical interface turns individual fingerprints into a 3D maze to wander through on screen; Community Connectivity, a workshop which presents instructions for building one's own wireless antennae; Botanical Gameboy, an installation of custom Nintendo gameboys powered by a network of lemon trees; and an interactive installation centered on the video game paradigm created by the student collective Artech. Transport, the online component of Playshop, will serve as a resource and window to the events and projects within the space consisting of a mailing list, database and gallery.

My favorite is definetely the gameboys powered by a network of lemon trees: the Botanical gameboy:

What\'s your personal video game history?

in

The IFTF is starting new research project on games: What's your personal video game history?.

To help us develop this history, we’d like to invite you to share your personal experiences with electronic games. Videogames are definitely included, but even earlier is fine. As long as some bits and bytes are involved.

So what did you play? What games stand out in your memories? Did you play in groups, or by yourself – did this change over time, or from game to game? What skills do you think you learned from each game? Has a videogame ever helped you in “real life”?

Send your personal gaming history, or as much as your time allows, to games (at) iftf.org

What's your personal video game history?

in

The IFTF is starting new research project on games: What's your personal video game history?.

To help us develop this history, we’d like to invite you to share your personal experiences with electronic games. Videogames are definitely included, but even earlier is fine. As long as some bits and bytes are involved.

So what did you play? What games stand out in your memories? Did you play in groups, or by yourself – did this change over time, or from game to game? What skills do you think you learned from each game? Has a videogame ever helped you in “real life”?

Send your personal gaming history, or as much as your time allows, to games (at) iftf.org

You Are What You Listen to

in

According to researchers from the Georgia Institute of Technology and Palo Alto Research Center (PARC), Co-workers sharing digital music in the workplace via Apple Computer’s popular iTunes® software form impressions of each other based on their musical libraries

“People sharing music in our study were aware of the comings and goings of others in the office because they noticed the appearance and disappearance of others’ music on the network,” said Amy Voida, lead author of the paper and a Ph.D. student at Georgia Tech. “They imagined what other people might think about their music collections, and they were aware of the musical holes left when someone left the company…. What once was an individual jukebox became a music community.” (...) “One of the greatest challenges for technical innovation in music sharing may be in allowing designers to make the leap between treating music sharing technologies as personal music listening utilities and treating music sharing technologies as online communities. Although music sharing has traditionally been a strong indicator of group identity and has reflected shared musical taste, our study of iTunes music sharing has demonstrated that even groups with disparate musical tastes can form strong group identities.”

More about this, in the following paper: Voida, A., Grinter, R.E., Ducheneaut, N., Edwards, W.K., & Newman, M.W. (to appear). Listening in: Practices surrounding iTunes music sharing. To appear in proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2005). Portland, Oregon, April 2-7.

Nabaztag: a WiFi rabbit as awareness tool

in

Nabaztag is a WiFi rabbit designed by Violet (a company focused on the design of products and services based on calm and emotional technologies). This rabbit can access the Internet. The most interesting part is that the colors change deppending on various parameters: the weather, car traffic or reception of emails. There is also different sounds as well as ear movement modified by those variables! It can also communicate with other rabbit located elsewhere thanks to a coded language you can create (like: a specific posiiton of the ear to show that you're busy...). 95 euros! It's designed by the same team who did the cool DAL, a wireless light that emits emotion via Wifi connection to the internet. I like this approach, calm technology is something of interest. I am wondering whether they make user experience analysis to understand more the usage of such device! This french company rocks. I strongly believe in this stuff. My concern is how to make user understanding the different kinds of information. How do they get it? Are there already any hacks or tricks (well the interaction is pretty low since it's not intended to offer lots of interaction but since it's wifi enabled there must be some tricks ;) ). I hope there next device will offer device-to-device interactions (with copresent rabbits). I imagine crazy scenarios with this.

Meeting with my PhD advisor

in

- keep doing the analysis: sequential analysis + nasa tlx + overlaps/backchannel/dispersion/covariate with bob's position + regression/multilevel modelling...- when looking at means differences, we have to be careful about the differences. For instance, a difference of 1 units (i.e. between 4 and 5) even significant must be stated carefully. The amplitude is not that big. - there might be an inertia caused by knowing where are the partners. - the performance measure in this task is the group path length. HOWEVER we did not really design the game to have a clear performance measure (this was just meant to find winners), the REAL performance if the accuracy/quality of participants' model of their partners' (intents, goals, activity). - the (planned) strategy, if well established imply less coordination on the field. AND coordination implies awareness of others OR messages. Now there might be a relation between the (planned) strategy and awareness/messages. Awareness...Messages... Errors BUT there are also link between awareness and strategy, awareness and message, awareness and errors. - That is why we need to discriminate the different strategies groups had (what I sketched on Pierre's whiteboard: three different group path + the fact that some groups stick to their strategies and others reshape it + when people converge to Bob, some go directly and others still wander around). - Discriminating people's strategies lead us to find descriptors: if we wanted to replay the games, which paramaters would we need? A simulation of this would imply to play with these parameters so that the agents use the same strategies as the ones encountered (emergence?): dispersion at the beginning + a certain speed to converge to bob. - we want to access to groups' strategies! Which indexes: angles made by the 3 persons? dispersions index (then have a graph: y=mean distance between the 3 persons, x=time) + we can define the EPFL as a graph/network amde up of different PLACES connected (like CO -> Archi, Elec, Unil, Esplanade....). So: 1) represent the EPFL as a graph 2) Draw players' paths (groups) thanks to 2a) path logs 2b) autoconfrontations, 3) Define categories (2/3) like for instance if they explore this graph in width or height: that may define strategy descriptors. - FIND literature about how can we describe people's trajectors: look at rat experiments in psychology. How can we analysed this?

- does the presence of the awareness tool trigger other/different strategies? What I saw is that people with AT stick to their strategy and the others modify it, reshaping it by annotating the maps.

- a mobile representation: strategy depends on: A's position, A's signal, B's position, B's signal, C's position, C's signal. In the condition with awareness tool, people have everybody's positions + potentially access to everybody's signal. In the condition without the tool, people have just their position + potentially access to everybody's signal. Agents use those inputs to build their strategies. We'll have to use them to simulate the task

Analysis of mold annotations

in

Another interesting and somehow improbable research project: Annotations of Prostheses (still at the University of San Diego):

The orthopedist annotated this mold of a polio patient’s right leg to represent areas from which to refrain from filing. This mold goes on with a form to a fabricator. The shape and markings on the mold, with the form, allow the fabricator to create a custom-made brace from patient-specific physical information.

The fabricator is preparing a mold to fit an orthotic brace, filing it to a shape similar to the patient’s leg. He is also providing information on where the orthosis should apply pressure for support, considering the texture of flesh at different points. The ink of the pen is well suited for deep penetration into the mold, avoiding loss of information through filing.

The reason why they are interested in this is that they work on annotations. They're point is that annotations are of communicative and cognitive importance (as aids in information movement, memory, attention, and organization). Now it would be nice to find sociological research about mold annotations practices!

Awareness and cooperative work in a café-restaurant

in

This paper (in french even though the abstract is in english) might be of interest, I'll peruse it in the train tonight: Awareness and cooperative work in a café-restaurant by Béatrice Cahour and Barbara Pentimalli. In Activités Vol. 2 No 1.

In this paper we present a workplace study of the collaborative activity between cooks and waiters in a restaurant/coffee-house. We show how awareness is linked to the attention mechanisms of the participants and how their level of awareness is constantly varying. First of all, using video footage, we analyse the functions of awareness between the co-team members through their peripheral attention, over-hearing and kinaesthetic perceptions: how they need to use shortcuts in their displacements and rapid communication so as to gain time, how they need to avoid collisions in a narrow space, etc. Secondly, based on 'explicitation' interviews conducted with two waitresses, we then present a detailed analysis of the variations in their attention which are linked not only to the cognitive overload present in their current activity but also to the objects and persons which are pertinent for this same activity. KEYWORDS: Levels of awareness, foci of attention, cooperative activity, socio-cognitive dynamics, communication, pluri-sensoriality.

Augmented video as a mechanism for improving collaboration and decision making

in

I stumbled across this research project at University of San Diego: Passing the Bubble:

We study augmented video as a mechanism for improving collaboration and decision making. Our special focus is decision making that depends on decision-makers and information analysts sharing their understanding. The interaction between these two groups involves commanders passing their intent to their information analysts, then refining their plans and decisions on the basis of information gathered by their analysts, often using AUV’s. We examine how different ways of augmenting video differ in how cognitively efficient they are in creating shared understanding. Most people who have seen augmented video assume it to be a powerful mechanism for communicating complicated objectives and facts about situations. But little or no work has been done on:
  1. determining which of the many ways of augmenting video are most effective, and
  2. developing a cognitive theory that explains why these different methods differ in their potency and cognitive efficiency. Video, if properly annotated, promises to enrich and reshape collaborative exchanges. Our goal is to understand how to maximize the impact such videos will have on collaboration.

Why do I blog this? Might be interested for my thinking about to use replay tool in my research project.