Icehopper is a simple game where the guest must jump and land on a green tile without touching the area in any of the other tiles. Initially the prototype was HMD, 1st-person based, but it was quickly discovered that a 3rd-person view was much easier. A simple themed environment and sound effects were added in the second iteration, along with a physical hopping device, VERTO.

Duration: 2 weeks, 1 full-time dev
Software used: Mocap Alice
Lessons learned:

• 3rd person does work better for the icehopper
  
• sounds and nice graphics make it much more fun
  
• vicon has problems at the end of the jumping trajactory (boot goes through the floor)
  

The body cursor tests simple raycasting in 3D space and appendage torquing. The guest controls a virtual hand and reticle to point at a ring of icons. When an icon experiences the 3D equivalent of a mouseover event, the center of the ring lights up with the selected icon. The ring of icons may be rotated by a subtle twisting of the wrist.

Duration: 2 weeks, 1 part-time dev
Software used: Mocap Alice
Lessons learned:

• having reticle is very useful, just having a virtual hand in the space gets in way
• pointing at things with your hand is very intuitive (and surprisingly fun!)

The Fogala is an experiment to generate original music via 3D body movement. Different timbres are selected by 2D movement on the ground plane (encoded by tracked, reconfigurable, laminated floor tiles) and note volume/chord progression are tracked by extension of the arms over the set of 3D planes.

Duration: 4 weeks, 1 part-time dev
Software used: PyGame
Lessons learned:

• having a background in music performance generally increases user enjoyment
• radial arm-control allows greater fidelity for pitch variation than vertical arm movement
• low latency is a must for performing real melodies
• dynamic instrument calibration allows greater guest freedom for expression
• having notes too close to one another causes problems, "dead zones" may aid in increasing performance
• people's expectations parallel that of an authentic traditional instrument
• having a lack of a clear goal issued to the guest in test can decrease enjoyment and instill a feeling of awkwardness

video only

audio and video

YMCA examines simple (head and wrists) body posturing. The guest is encouraged to form one of four poses. If the pose is matched correctly within a certain tolerance, a congratulatory sound is played along with positive visual feedback.

Duration: 3 days, 1 full-time dev
Software used: Mocap Alice
Lessons learned:

• traditional (MasterMotion) pose matching that uses 3D spatial relation works well for this application (whole body approximation to an actual pose works)

Dance Dance Anubis is an evolved form of YMCA that employs both the body cursor and a refined hand-and-foot pose matching grammar. Guests are enticed to walk like an egyptian and form reflective images of animated hieroglyphics.

Duration: 2 weeks, 1 full-time dev
Software used: PyGame
Lessons learned:

• designing for the least-skilled user works, but perhaps too well
• pose-matching in anubis worked much easier when projecting onto a 2D plane with a constructed pose from simpler components
• a seamless audio and visual interface to an experience greatly increase guest enjoyment
• people didn't look at the representation at self, or understand its utility -- whether because of actual infidelities or preference to focus on the goal
• guests did look at the representation of self primarily for left/right color matching
• this was difficult to perceive on the 2D surface (especially for feet)

Ghostbusters is the second of two hybrid prototypes which relies on both the ray-casting of Klingon UI and the virtual object tracking previously devised in Wizards&Lizards. The guest uses a markered prop to ensnare hovering spectre and slowly guide him to the trap.

Duration: 2 weeks, 1 full-time dev
Software used: Mocap Alice
Lessons learned:

• a strong concept can help guests see beyond a simple implementation
• an overly simple implementation can limit enjoyment
• HMDs are still a poor limitation for immersive VR
• physical props and themed real-world effects greatly increase enjoyment and immersion
• in a story/familiar concept-based application, guests have a greater expectation for traditional narrative elements (raising of stakes, multiple stages, charming characters etc.)

Mocap Ghostbusters in-game screenshot.

Building on the well-received body cursor, our next evolution for real-time mocap was the body as a giant joystick. Using a popular Nintendo Entertainment System Emulator and some ROMs, various activities were mapped to the body on a game-by-game basis. Now, operations such as jumping in an action game are physically mapped to the feet leaving the ground in the motion capture space. Running left and right controls horizontal avatar manipulation, and a wide array of firing and launching features are bound to the movement of the arms. Although the fidelity for many of the games is not what it is with the traditional thumb-controller, it is, however, often a lot more fun (and exercise).

Duration: 2 weeks, 1 part-time dev
Software used: Python, Java, FCEU
Lessons learned:

• Workout games are a lot more fun if the game doesn't let on that it's a workout.
• Our spatial partition method of checking poses works very well and is flexible.

After experimenting with the note-based Fogala, the next step for the project was to play with the instrument's mixing capabilities, coupled with visualisations for multi-purpose feedback. The Veelom consists of a set of loops and drum patches that may be activated by moving one's hands through the motion capture space. Volume is controlled on the vertical access and loops/patches are laid out in virtual row-like arrays over the floor. The guest's hands' position is displayed on the forward-facing projection screen. The screen also serves as a menu that is manipulated by the popular body cursor in addition to allowing real-time 3D visuals.

Duration: 3 weeks, 1 part-time dev
Software used: C, C++, Python
Lessons learned:

• top down view is helpful after users become acclimated
• flashy gui impresses people, increases quick appreciation
• reusing the mixer saved a lot of time
• some users missed the freedom of individual note playing that the fogala allowed

Continuing with the theme and functionality of the original, Dance Dance Anubis 2 moves the game into a 3D world created in LithTech Jupiter, while retaining the game's general concept and theme. A larger variety of poses were adopted that when performed in sequence, more closely emulates actual dance by the fluid movement of the guest in time. As opposed to simple pose matching, the core of DDA 2 is hitting poses on the beat in choreographed succession.

Duration: 4 weeks, 3 full-time devs
Software used: C++
Lessons learned:

• Pose matching can be really sloppy and still be fun: two different pose cards may actually have the same pose under the hood, but as long as the game responds as soon as the player steps into the one she think she's trying for, she's satisfied.
• A fully animated "This is you" representation of the player in world isn't always necessary; sometimes people respond just as well to schematic floating dots.
• Our spatial partition method of checking poses works very well and is flexible.