sensory substitution

I’m collaborating with the Philosophical Psychology Lab at Gordon College, working with philosopher Brian Glenney on experiments with the vOICe, also known as the Seeing With Sound device. The tool utilizes a camera, embedded in a pair of glasses or goggles, and translates its intake into “readable” soundwaves through earphones—either assigning sounds to light and dark values, or, in a modified version by lab member Zach Capalbo, into the basic color spectrum.

The device is designed with practical use in mind: as an adaptive tool for those who are blind. Glenney and his team are interested in its capacity to blur our normally discrete senses—activating the visual cortex with soundwaves, for instance. And we both got interested in the device as a tool for augmented reality—what kinds of experiences it makes possible that are otherwise absent in our typical sensory hierarchy?

After first getting some basic training in translating the sounds, I learned to perform a basic light-dark distinguishing search (as in the photo of me in this web site’s masthead). And then I participated in this two-person search game. Not since childhood had I played this kind of hide-and-seek-in-the-dark.

We each had red-lit hearts attached to our chests on backpacks. We could turn the lights off and on by listening to the location of our opponent, and lunging for the heart-light once in proximity.

This was just one of the experiments that ultimately led to our exhibit of drawings and video, “I Never Asked to Be Made Human,” now on display at Gordon College. More documentation and project discussion to come.

2 thoughts on “sensory substitution

  1. This is incredibly enlightening! That our interaction with the world can be affected by transmogrifying different senses, one to another “seeing” and “hearing” seems to gain complex meaning through it all, challenging our presumptions about what it means to see. I’m fascinated. And it’s ironic that it’s technology, something expected to heighten our individual senses, should be giving us a chance to explore our inter-sensory depth! Please keep it going and keep me informed….

  2. What can be done to make the device more “organic”, more “biological”. Thanks Sung for the point about it being technological, but I’d like for it to become technorganic (what a terrible word).

    Sara drew up a couple basket-weave designs for the goggles. Is there some way to push this organic quality?

    Sara, any speculative goggle designs?

    Our latest goggles have a antique white (skin white?) color and the camera lens pops out of a hole I drilled in the lens. I’m out of my lab, but I’ll send pics soon. Anyway, this is an attempt (a failed one) to be more organic.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s