Language, Dolphins and Garage Cinema

What if dolphins communicate by sending and receiving images? What if humans can learn to do the same, on the fly, via computer mediation?

I know what you're thinking: this guy's been in California far too long. You're probably right.

But bear with me on this.

As a kid growing up by the sea in Florida I was obsessed with bottlenose dolphins. I read everything I could find about them. When I was 13 I borrowed a fancy underwater microphone from an oceanographer and used it to record dolphin sounds at Ocean World, the local marine theme park. I played back the recordings into another dolphin tank. But I didn't get much of a reaction at all. After gathering around this strange noise-making machine for a few minutes, the dolphins quickly grew bored and ignored the tape recorder. They were far more interested in my cheap watch. And dead fish.

Plenty of more serious research (and writing and movie-making) was devoted to the prospect that dolphins' clicks and whistles might be a language. Scientists showed that dolphins convey instructions to one another, but still nobody has proven whether a high-level dolphin language exists.

This week I read a paper by Berkeley's Professor Marc Davis that dramatically changed my thinking about this by pointing out that a dolphin language might not be based on words. Most linguistic dolphin research I've seen seeks dolphin sounds strung together as words, and I always unconsciously assumed that any high-level language must be based on words.

But Davis points out in his paper that some written human languages don't use words at all but instead directly represent meaning visually. The image to the right is a message written in such a language, by a member of the Yukaghir tribe in Siberia. (See Davis' paper for more details and a translation of the letter.)

Like bats, dolphins use echolocation. They emit waves of sound and use the resulting echoes to pinpoint locations, sizes, shapes, densities, and even internal states and structures of animals and objects, with astounding precision and accuracy. If dolphins use their own sounds so skillfully to probe their environments and to "see" what's around them, can they also use sound to create artificial imagery that's "visible" to other dolphins?

Dolphins exhibit a superhuman ability to convey spatial instructions to one another. Nobody's sure how exactly they work this out, but if you watch a group of dolphins carrying out tasks in which they have to quickly synchronize very complicated sets of movements -- during a theme-park performance, for example, or during hunts in which they round up thousands of fish into dense schools -- you'll be amazed at their powers of spatial coordination.

Can you imagine dolphins sending each other visual cues mapped to real-world environments -- or even sending entire artificial "video" scenes showing planned activities -- on the fly?

This may be a stretch; it's probably fiction and so far it's not backed by much science. But it's a very powerful idea that we can use. Even if dolphins cannot communicate this way, perhaps we will be able to, with the help of computers.

Davis and his Garage Cinema Research group at Berkeley are working on it. They're designing systems that they hope will allow regular people to easily and quickly build video compositions, without putting forth the tremendous amount of time, expense and technical knowledge necessary for today's film production. Thanks to smart systems that can recognize media assets and automate much of the video capture, editing and production process, Davis hopes to allow us all to "write" video as often and as easily as we "read" video today. The promise lies not just in replacing the current wasteful and corporate-dominated system of creating polished high-end feature films, but in providing humanity with a new, more powerful form of everyday communication.

Posted by sean at May 11, 2003 06:32 PM | TrackBack (0)