printer friendly version of article
transference : artificial perception
View this article in flash requires flash 6 >
Artificial Perception as Reality Check
Thinking about MIT's Tangible Bits
by Tom Sherman
David Rokeby isn't the only interactive designer working with innovative human-computer interfaces. Certain creatively-inclined computer scientists are also leading the way. In a future envisioned by engineers, we may live surrounded by "interactive surfaces" that turn everything from walls and desktops to landscapes and air into digital inputs and displays. Tom Sherman asks some tough questions about what will happen to human cognition in a world of ubiquitous technology in his review of MIT's Tangible Bits Physical Interface Project.
Tangible Bits is an attempt to bridge the gap between cyberspace and the physical environment by making digital information (bits) tangible.
-- Hiroshi Ishii, from his Web site
Hiroshi Ishii started the Tangible Media research group and their ongoing Tangible Bits project in 1995, when he joined MIT's Media Laboratory as a professor of Media Arts and Sciences. Ishii relocated from Japan's NTT Human Interface Laboratories in Kyoto, where he had made his mark in Human Computer Interaction (HCI) and Computer-Supported Cooperative Work (CSCW) in the early 1990s.
I met Ishii when I visited his lab in 1997 while conducting research for Ars Electronica's FleshFactor symposia and exhibitions. At the time, there was (as there still is) a lot of new renaissance hype coming out of MIT. But despite what anyone may have heard to the contrary, engineering still rules at MIT. To Ishii's credit, he doesn't pretend to be an artist. His research focus has always been on the design of seamless interfaces between humans, digital information, and the physical environment. Ishii is an engineer interested in perception. That said, his use of written language to over-state the creative dimensions of Tangible Bits research sometimes verges on poetic hyperbole.
By 1997 there was a steady stream of rhetorically sophisticated "literature" pouring out of the Tangible Media lab. The story begins with the shortcomings of computer interfaces to date. Graphic user interfaces (GUIs) -- screens and keyboards and mice -- prohibit people from using their higher, natural skills for sensing and interacting with their physical environments. Computers are currently anti-body. You can't touch the data you are working with, or use your body to move around it. But some day computing will be more accommodating to multiple intelligences -- including bodily/kinesthetic and musical/rhythmic intelligences. Tangible Bits seeks "to build upon these skills by giving physical form to digital information, seamlessly coupling the dual worlds of bits and atoms." The idea of a "seamless" integration of digital language and devices into the physical domain is a central theme.
One strategy for eliminating the "frame" separating computing from the rest of world is to spread digital information into the background. Ideally, hands-on, foreground interactions with computers will be informed by information lingering at the periphery of the user's senses. The Tangible Bits philosophy is anchored on the gestalt theory of Max Wertheimer. In all learning environments, context is important. According to the pervasive-computing scenario of Tangible Bits, in the near future we will live surrounded by things such as "interactive surfaces, whereby walls, desktops, ceilings, doors, windows, etc. become an active interface between the physical and virtual worlds." The rooms we live and work in, the cars we drive, the terrain, vegetation, and water we encounter, will all eventually yield digital information. Ishii's group "is seeking ways to turn each state of physical matter -- not only solid matter, but also liquids and gases within everyday architectural spaces -- into interfaces between people and digital information."
These lofty goals have been substantiated in the somewhat primitive technical achievements of the Tangible Media Group to date. Throughout the 1990s and into the present, Ishii and his research associates (mostly MIT graduate students) have typically demonstrated half-a-dozen "tangible interface" prototypes a year. Their projects have resulted in curiosities like Audiopad, a real-time electronic musical instrument comprised of movable pucks on a flat display surface. Also, Illuminating Clay, a computational landscape-modeling system featuring digital graphics projected over malleable putty. Without diminishing the difficulties of trying to close the great divide between atoms and bits, these devices are clearly master's thesis-sized projects in terms of achievement, and appear to be baby-steps in a rather gimmicky research field. My personal favourites include musicBottles, wherein different parts in a musical arrangement are "played" by removing the caps from three transparent containers -- this project is said to exploit "the emotional aspects of glass bottles." Also, LumiTouch, a pair of picture frames networked so that they light up when long-distance lovers hold photographs of each other. The shallowness of these touchy-feely attempts to communicate emotional content only serve to undermine the Tangible rhetoric.
The vulnerability of this research is its extreme literalness, its nuts and bolts lack of poetics. It is ironic that these hardware-based prototypes serve to deconstruct and demystify, rather than to strengthen, some of the group's best claims. Tangible Bits research is conducted by computer scientists and students in interdisciplinary teams (different species of engineers, cognitive psychologists, and so on). The profiles of these researchers generally reveal parallel, hobby-like interests in music and the visual arts, plus lots of hiking, camping, wind-surfing, and yoga. It's clear that being creative and pragmatic, killing two birds with one stone, is an art form in Ishii's lab.
With the above criticism levied, it is hard to argue against the wisdom of developing dual, or multiple-purpose systems. And all these modest, thesis-level projects will eventually accrue into a significant engineering domain. MIT attracts brilliant scientists and students, and I have no doubt that there is more behind the Tangible Bits initiative than meets the eye. Just look at the wonderful promises.
Max Wertheimer said that we should seek to discover the underlying nature of things (the relationships between elements, both figure and ground). Ishii is a gestaltist, and he learned a great deal from the late Mark Weiser, the brilliant former chief technologist at Xerox PARC. Weiser launched the idea of ubiquitous, or pervasive, computing. Mainframes gave way to personal computing, and computing will now move out into the physical environment, in what Weiser said would be an era of "calm technology" In other words, technology will recede into the background.
Hiroshi Ishii has distilled Wertheimer and Weiser's thinking into the key research goals of Tangible Bits: develop interactive surfaces, couple atoms and bits (so that the surface of physical objects will reveal digital information), and move digital information into ambient media. (Background interfaces at the periphery of sense perception are absolutely key: again, context matters.)
The extreme literalness which typifies the way engineers apply perceptual theory leads me to predict the next twist in Ishii's rhetorical narrative: If human perception depends entirely upon information in the environment (the Tangible Bits vision is a literal projection of the act of perception onto the environment), then the way we colour or distort the world in our internal cognitive processes can be over-written and ignored. Advertising agencies will love the idea of living rooms where every single surface reinforces a pitch!
Perception in Ishii's model will end up being a direct consequence of the properties of the environment. The imagination, and our "memory" of prior learning, will actually be composed by the environment. We will slip into a sub-symbolic reality: a childlike state of sensual reverie. Rhetorical substantiation for such a vision may be obtained from two texts by J. J. Gibson: The Senses Considered as Perceptual Systems (1966), and The Ecological Approach to Visual Perception (1979). Gibson, in his theories of ecological psychology, stressed the importance of interaction: give and take between the organism and the environment. Active, physical learning, where major information is picked up by moving around and finding out what happens, is the guiding principle of Gibson's thinking.
There's no question that today's GUIs pin us down, immobilizing our bodies, restricting our computing environment to a symbolic, physically inactive space. But what will happen to our internal, cognitive processes when we start slipping in and out of cyberspace by physically moving around: walking, running, jumping, bumping, and caressing? This is where the passion of engineers who love to hike and bike, windsurf and practice yoga, comes into play.
Tom Sherman is an artist and theorist who splits his time between
Port Mouton, Nova Scotia, and Syracuse, New York, where he teaches media art history, theory, and practice at Syracuse University's Department of Art Media Studies. His latest book, Before and After the I-Bomb: An Artist in the Information Environment, was published by the Banff Centre Press in 2002.