Skip to content

Song Searching

Sirovich Family Student Fellow – Ami Cai
STEAMplant Faculty Members – Christopher X. J. Jensen, Basem Aly, Jennifer Telesca
drawn art of water waves

Humpback whales are some of nature’s most majestic creatures. These immense marine mammals undertake epic annual migrations, feed cooperatively, form local cultures, and interact socially using songs transmitted over thousands of kilometers. What would it feel like to be a humpback whale experiencing today’s increasingly human-dominated world? “Song Searching” is a video game designed to give players the experience of being a humpback whale. To succeed in the game, players must successfully migrate through an environment filled with large ships emitting noise to arrive at their feeding and breeding grounds. Players will navigate the environment using both visual and audio cues, with stereo audio perception being a key guide to play. 

There are plenty of video games that immerse players in fantasy worlds, creating experiences that foster emotional attachments to fictional characters. There are also plenty of documentaries and lectures that stimulate knowledge of threats to whale populations. This project seeks to blend knowledge from both these realms to create a unique way for people to both empathize with whales and to understand the threats posed by human activities to marine life.

Pratt Research Open House (2019)

Since sound travels roughly four times faster in water than in air, and the farthest the vertebrate eye can see is only roughly 100 feet, whales have evolved to have amazing hearing rather than high-acuity eyesight. Humpback whales are one of the few species of baleen whales, a subcategory of whales that sing songs to communicate with one another. Commonly found only in males during migratory seasons, there are several theories of the purpose of the complexities of these songs, but various usages are assumed for social interactions, signaling danger, mating, or as a way finding tool to navigate to feeding and breeding grounds. The game explores the possibility that whales are using their song to navigate to these important social areas. In order for the player to reach its companions during critical migratory seasons, they need to complete the course before the end of the time limit, ranging from a few minutes to 20 minutes depending on the level.

To mimic what the whales are able to hear, the audio pans between the left and right speaker sets. The audio of songs from other whales will be the player’s wayfinding tool to navigate the terrain and make their way to feeding and breeding grounds, while other sounds forewarn of incoming obstacles and interfere with finding other whales. Common sources of noise pollution include commercial vessels, recreational boats, air guns, military sonars, underwater construction, and seismic surveys. The game play is in 3rd person in a survival/dodging style gameplay avoiding obstacles representing the various sources of noise pollution. There are a handful of different levels that parallel the history of industrialization and the globalization of large boats and other sources of deep-sea noise. As the player progresses from the past into the present, it becomes more and more difficult to navigate the waters due to the increase in noise pollution. There will be an increase in the number of obstacles but also an increase in the radius of sound around each obstacle.

Click here to download the full project PDF.