Ambient Communication

With the advent of location tracking services such as Loopt, we saw a need for a means of location awareness that was a bit more privacy preserving. Clearly there was a need for systems that helped people coordinate and plan in-person get togethers, but these needs were not well known. We designed a study to listen in on people’s phone calls (with permission from all parties) and we focused on their use of location or activity words in their communication. With this data, we were able to deeply understand why people share location with each other and in what circumstances they do so. Key findings indicated that people in close social relationships often know the basic patterns of locations for their friends throughout the day, but are unsure of the exact transition times (e.g. has someone arrived home after work by this point). We also observed how background noises and context in a phone call can indicate location in a more passive way. All of these cues and instances of locations sharing add up over time to create a rich portrait of a friend or family member’s life. With this data, we created a long list of potential concepts based on the behavior we observed. These concepts served to protect users’ privacy by not sharing absolute location information and hoped to increase trust and allow for more spontaneous meet-ups with less planning and without the overhead of phone communications. The Motion Presence concept was explored in more detail and is discussed below.

Publications:
Location and activity sharing in everyday mobile communication. Frank R. Bentley, Crysta J. Metcalf. CHI EA ’08 CHI ’08 extended abstracts on Human factors in computing systems, 2008

Using the findings from the Location Sharing study, we created a system called Motion Presence. This system shared one bit of information: if contacts were currently at a place or moving between places (and how many minutes they were in that state) in the phone book on a mobile phone. By not sharing absolute location, we hoped to preserve privacy since interpreting the data requires previous social knowledge. We conducted a field evaluation of this system with couples in various states of relationships as well as groups of friends to see how it could fit into their lives. Participants used the new phone books to determine availability (e.g. has someone arrived at work?), get more time at an activity (e.g. taking longer to get ready when they noticed that a friend was running late), confirm someone’s safety (e.g. confirming someone has started or stopped moving when they are expected to). The Motion Presence concept taught us the great power of a single, simple piece of information in helping people coordinate and learn about their friends and family in new ways without needing explicit synchronous communication.

Publications:
Sharing motion information with close family and friends. Frank R. Bentley, Crysta J. Metcalf. CHI ’07 Proceedings of the SIGCHI conference on Human factors in computing systems, 2007

The Use of Mobile Social Presence. Frank Bentley, Crysta Metcalf. IEEE Pervasive Computing, vol. 8, no. 4, pp. 35-41, Oct.-Dec. 2009

A time to glance: Studying the use of mobile ambient information. Frank Bentley, Joe Tullio, Crysta Metcalf, Drew Harry, Noel Massey. Pervasive 2007 Workshop on the Design and Evaluation of Ambient Information Systems. May, 2007.

The Music Presence concept came from our Music Context study. We observed how people have their go-to sources for new music and how the music that particular friends are listening to spreads throughout a social network. At this same time, we were working on the Ambient Communications program of research, and wondered how currently-playing music metadata could be shared in the background throughout the day with close friends and family members. Our summer intern, Drew Harry, created the initial prototype which was quickly built using SMS and the Audioscrobbler service. It allowed for friends to receive SMS messages containing the artist and song title of each song played by their friends. At any point, users could open up their SMS inboxes to see a history of the music their friends had been playing. (Participants were encourages to turn of their SMS ringers for messages from our account so as to be less intrusive) This rapid prototype allowed us to quickly field the concept in real settings over a one week period to see how this music information could be used in daily life. We observed users explicitly playing music for others to see, users learning about the music tastes of others, users making fun of other’s music tastes or gaining a new respect for others based on their music choice, and using music playing information to know that friends were bored and available on a Friday night. Concepts from this work are now visible in the Contacts 3.0/MOTOBLUR system which has Last.fm social music integration as well as in the Motorola Connected Music Player.

Publications:
The Use of Mobile Social Presence. Frank Bentley, Crysta Metcalf. IEEE Pervasive Computing, vol. 8, no. 4, pp. 35-41, Oct.-Dec. 2009

A time to glance: Studying the use of mobile ambient information. Frank Bentley, Joe Tullio, Crysta Metcalf, Drew Harry, Noel Massey. Pervasive 2007 Workshop on the Design and Evaluation of Ambient Information Systems. May, 2007.

After completing the Motion and Music Presence systems and working with TileFile, we saw the opportunity to create a general-purpose ambient information delivery system for the next generation of the phonebook on Motorola devices. Joining with a design team from a product group, we created Contacts 3.0, a service for integrating status updates, photos, and other contextual information about contacts in the phone book application on the device. This served as a key opportunity to get Motorola into the services business as well as a transformative way for users to learn about their friends and family. We moved away from the siloed application paradigm then common, where a user would have to launch Facebook, then return to the home screen and launch Twitter, then return again and launch Flickr, etc. to see updates across services. We tied all updates to an identity in the phone book, providing a single place to visit to learn about friends and family. In this way, the phone book became a desitination to learn about contacts instead of just a place to dial numbers or look up addresses. Contacts 3.0 shipped as the default phonebook on phones equipped with MOTOBLUR, with tens of millions of users. It was also the focus of Motorola’s 2010 Super Bowl commercial.

Publications:
Contacts 3.0: bringing together research and design teams to reinvent the phonebook. Frank R. Bentley, JoEllen Kames, Rafiq Ahmed, Rhiannon Sterling Zivin, Lauren Schwendimann. CHI EA ’10 Proceedings of the 28th of the international conference extended abstracts on Human factors in computing systems, 2010

Patents:
Aggregated View of Local and Remote Social Information. Frank Bentley, Rafiq Ahmed, JoEllen Kames, Lauren Schwendimann. US Patent Application Number:20090209286.

In 2007, we began working with TileFile, an Australian startup with a Flash-based photo-sharing website, to create a mobile experience for their service. TileFile had a concept of photos and videos as “tiles” or surfaces with a front and a back. In the same way that one can write on the back of a photo, the back of a tile contained the metadata for that tile including the title, time and place of capture, etc. In a week-long workshop, we met with the TileFile team to design a mobile interface for their service and begin implementation. We continued the collaboration through the implementation phase and ran a three-week field evaluation of the system in Chicago. From this study, we learned about the power of real-time delivery of photos on a mobile platform to break down distance and be visually connected to a friend or family member’s life through both mundane and special moments in life. We also gained first-hand experience with systems that store both public and privately shared content. TileFile was released as a public service for phones with J2ME and UIQ, however the world quickly changed to iPhone and Android leaving this service behind as other photo sharing sites quickly created mobile applications. We used insights from this field study in later work, such as Contacts 3.0 and StoryPlace.me.

Publications:
The Use of Mobile Social Presence. Frank Bentley, Crysta Metcalf. IEEE Pervasive Computing, vol. 8, no. 4, pp. 35-41, Oct.-Dec. 2009

For my masters project at the MIT Artificial Intelligence Lab in 2002, I created a system for what we termed Perceptive Presence applications. The middleware enabled applications to be written for Perceptive User Interfaces (PUIs) in a very similar way to Graphical User Interfaces (GUIs), removing much of the overhead to working with low-level camera drivers and machine vision code. I created widgets for detecting various gestures and spatial selection operations. Using this middleware was no harder than creating a GUI with standard Java UI widgets. On top of this infrastructure, I created the Perceptive Presence Lamp. This lamp virtually connected two spaces (e.g. offices, homes, etc.) and conveyed a sense of the activities in the remote location. This could be used to better manage collaborations or as an aging-in-place tool. The lamp in the first location would be off if no one was present in the second location. It would turn green if one person was present or red if multiple people were present or conversation was detected. A user could get the attention of the person in the other location by staring at the lamp for several seconds, in which case both lamps would flash blue, similar to staring at an officemate for a few seconds to get their attention. If the remote user then stared at their lamp, an audio connection would be opened between the spaces. This system explored new ways to be aware of people at a distance through the use of colored light. The cues from these lamps could then be used to better collaborate over a distance or to be aware of loved-ones who might need additional attention.

Publications:
Perceptive Presence. Frank Bentley, Konrad Tollmar, David Demirdjian, Kimberle Koile, Trevor Darrell. Computer Graphics and Applications, IEEE , Volume: 23 Issue: 5 , Sept.-Oct. 2003. Page(s): 26 -36

Face-Responsive Interfaces: From direct manipulation to perceptive presence. Trevor Darrell, Konrad Tollmar, Frank Bentley, Neal Checka, Loius-Phillipe Morency, Ali Rahimi and Alice Oh (MIT AI Lab), UbiComp 2002 Proceedings

In 2006, another team in our lab was working on a social television system. They had developed an on-screen user interface to display a friends list and to display the current channel and program that each friend was tuned to. In addition, there was the ability to chat with contacts or open up a live audio connection between houses. Users could also invite friends to join them in watching a show together. However, since (even in America) televisions are not on at all times, there remained a problem in knowing when friends were watching television or wanted to invite you to watch something together. I joined the project to create an ambient interface that would be active even when the television was off. Using work from my masters project on Perceptive Presence, I used an Ambient Devices color-changing orb to convey social information even when the television was switched off. The orb was off when no friends were currently watching live TV. It would change colors as more friends tuned in, transitioning softly between colors so as not to be too distracting. At a glance though, people in the room could see if something popular was on that all of their friends might be watching. If a friend invited the household to watch a program together, the orb would pulse softly, enough to notice, but possible to ignore if the household decided not to turn on the TV. In addition to the orb which sat near the main TV, a Chumby was placed in the kitchen or other common area to display the same information. We ran a two-week field evaluation of this system with two sets of five households that knew each other. We observed how the ambient displays drew members of the household into turning on the television, checking presence, joining friends in watching a program, and then chatting with friends about the show. This ramping interaction, that started with a simple presence cue, created memorable experiences watching and conversing around content for our households.

Publications:
Ambient social tv: drawing people into a shared experience. Gunnar Harboe, Crysta J. Metcalf, Frank Bentley, Joe Tullio, Noel Massey, Guy Romano. CHI ’08 Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, 2008

Examining presence and lightweight messaging in a social television experience. Crysta Metcalf, Gunnar Harboe, Joe Tullio, Noel Massey, Guy Romano, Elaine M. Huang, Frank Bentley. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), 2008

Patents:
System and Method for Providing Status Information of a Multimedia Broadcast Receiver on an Ambient Device. Frank Bentley, Noel Massey, Joe Tullio, Gunnar Harboe, Crysta Metcalf, Guy Romano