Low Attention Spatialized Auditory Entertainment
Interface for a DMC Delorean


Mission Statement

Create a low attention audio-visual-haptic interface for browsing, selecting, and personalizing audio entertainment for the interior of a Delorean.

Interface Description

Our proposed interface uses a haptic touchpad to navigate a 3x3 simultaneously streamed audio matrix. The audioscape is supplemented by a small video monitor located in the center console, haptic feedback thumping via a solenoid, and a mutable voiceover. The nine element matrix consists of two labeled axes, which can be customized by the user. Each of the nine audio samples exists in its own isolated space within the interior. The currently highlighted sample has an increased volume relative to the other samples in the matrix. Selections are registered by tapping the touchpad, resulting in an "audio zoom" into the next level of the hierarchy. Eventually the guest will reach a final selection, and this selection will amplify and expand to replace the browsing matrix. Additional features are accessible via a "glyph mode", activated by a nearby rocker switch.

Scenario

Bryan is preparing to leave his house and plans to use his car for some errands and then eventually arrive at work. As he gathers his belongings in preparation to go, he presses a button on his key chain which will undock his car from its network docking station in the garage and change over it's power supply to the battery of the car. The car is customized with the data he needs to make his travels both enjoyable and productive: the hard drive is loaded with his favorite MP3's and his email and phone messages will arrive to his car while it is active on the network.
Bryan says goodbye to his family and makes plans to be in touch with them later in the day about plans for the evening. He will also need to get messages about the kids carpool plans since he might have to pick up Liza and Helen after swim practice. The schedule is up in the air and final plans will need to wait until later.
Bryan unlocks the door, sits down and starts the engine of the car. As he is settling in, a small screen in front of the car displays five main categories of interactive car controls: climate control, audio, personal communications, navigation, and vehicle diagnostics. A touch pad interface, which is located between the two front seats, acts as the selection and navigation device for the interactive controls within the car. At the top level of the interactive car controls, the touchpad and the screen have a one to one spatial mapping which eases wayfinding through the menu selections. Bryan double taps on the touchpad to select the audio control. The car contains four speakers which can overlap and spatialize sound into 9 discrete locations within the interior of the car. The touch pad is used to scroll through the audio landscape via a 3x3 matrix window overlay (a kind of magic lens.). The scope and scale of the audio landscape is pre-configured by the driver at setup time.
Not sure of what he wants to listen to, he navigates the song selections via the touch pad. As his finger scrolls around the 3x3 matrix, he can browse along an x and y axis which contains chunked data such as musical genres like jazz, classical, techno, etc, as well as style, gender, emotion, tempo, etc. These axe s are bi-directional and allow the user to make audio selections using a broad range of entry points. For each category he browses, Bryan hears a voice over which announces and confirms his selection. He chooses slow jazz as his top level categories and drills deeper to choose sub categories of jazz and eventually will choose and artist and their songs. As he is moving his finger around on the touch pad, he is able to hear the sounds of the music overlapping within nine locations of the car. Each time a selection is made on the touch pad interface, the selection on the screen is highlighted and magnified by 25 percent. This sound is brought to the foreground via an increase in volume but remains in its original location in the car. Each time he makes an audio selection by double tapping on the touchpad, the selection is moved to the front left corner of the car and he is able to drill deeper into his data set. Bryan continues and selects an artist and eventually a series of songs to play. While he is making the selections, the small screen is displaying the exact names of the selections within a similar 3x3 matrix.
Bryan has left the driveway and is driving down the road on his way to his first errand. He continues to select music for the rest of the trip. Each time a selection is played, a frequency counter passively collects information across the board about category, subcategory, tempo, artist and songs. This information contributes to the "favorites" collections, which are unique to each driver and could be called up at any time via a shortcut from the driver via touching a rocker switch above the touch pad interface.
Once the shortcut rocker switch is pressed and Bryan is in shortcut mode, he can enter a gestural glyph to immediately navigate to a personal musical collection. Since he has many different musical tastes and different passengers to accommodate, he often uses his glyphs to sidestep the music browsing selection process.
Bryan can randomize the musical selection at any level by pressing the rocker switch and entering glyph mode. After drawing the Randomize glyph, the musical selection can be randomized on either the x-axis (category, sub-category, artists, artist) by drawing a horizontal line, the y-axis (tempo) by drawing a vertical line or on both by drawing a diagonal line across the touchpad.
As Bryan is completing his errands, his mood changes and he selects new music to listen to; something calmer after he has been hassled at the bank, pet store and dentist. He is now on his way to work and when he gets to the parking garage, he will be able to dock his car so that it will upload the last bits of current data to his office where he can access the audio collections as well as his PDA notes, email and phone messages which have evolved since his car last uplinked its data to his server. Later, when Bryan leaves work, the data he generated will download to his car and be available to him as he drives home. This could become especially useful if he has to wait for Liza and Helen at the swim practice.

Technology

Hardware

Software

We are using Python as the core of our development environment. PythonTk was recommended as an interface toolkit for rapid prototyping. Our low-level C++ code communicates with Python via the Component Object Model (COM). There are two COM modules in the project. One retrieves the music data from a prototype Access database, and the other sends the decompressed music streams to the Environmental Audio Extentions (EAX) library. The database is designed to record statistics of the user, as well as provide a place to store the song metatdata. The schema diagram follows:


Tasks and Assignments

Bridget
  • Design visual interface.
  • Design glyph defaults
wil
  • Integrate Database with Python
  • Setup Tools
Mauro
  • Integrate EAX with Python
  • Hardware integration with Touchpad


Final Deliverable

We will demo a fully functional interface in the Doherty Lab using the existing game machine hardware. Delorean installation may occur with exterior mounted hardware, but a full installation cannot be accomplished in the time remaining due to part ordering and resource allocation.

Literature

"Explaining Spatial Problem Solving in terms of Cognitive Load or Responsiveness and Selective Attention" by Kay Owens

"Sonification Report: Status of the Field and Research Agenda Prepared for the National Science Foundation by members of the International Community for Auditory Display" by Gregory Kramer, Chair; Bruce Walker, Project Coordinator; Terri Bonebright; Perry Cook; John Flowers; Nadine Miner; John Neuhoff

"Design of Spatialized Audio in Nomadic Environments" by Nitin Sawhney and Chris Schmandt