Seeing non-verbal information with the “Total Visualization Tool” and the “Self Translation Machine”: MITSUO ISO

January 21, 2015

Animator and screenwriter Mitsuo Iso offers Part 2 of his three main essays on the theme of "I imagined the year A.D. 2036." Bodily information that cannot be verbalized—what future for jobs would we see if we could visualize this information?

There are new discoveries to be made from bodily information that cannot be expressed in words.

Let's suppose that, in 2036, a converter engine that can visualize the four senses other than sight, as well as various other data, has been invented. There is already a real-world example of visualizing the sense of smell to develop new fragrances. I wonder what we will be able to do once we've visualized the sense of touch.

It helps me think when I feel the texture of paper with my fingertips. So, for instance, you might look at the touch sensory data from during a meeting afterward and discover some strange peak in the data. Once you check the log of yourself speaking, you discover that you were talking about a good idea that you didn't realize at the time.

I draw pictures, so I've had similar experiences. There is a whole lot of information that your body is telling you. But your brain usually just discards it. I've actually experienced looking at the picture I've drawn and thinking, "Oh, so that's what I wanted to do," and I’ve learned things from my own hands.

Athletes say the same sort of thing. But, besides drawing, if we could comprehensively visualize touch and hearing—as well as things like your pulse and brain waves—we might discover bodily information that we weren't previously aware of.

I think we often have amazing ideas in our heads that just crumble to pieces when we try to explain them. I think there are caches of non-verbal images buried in human brains and bodies like a natural resource. I wonder if we couldn't uncover these hard-to-verbalize ideas by visualizing our own unconscious minds. You could also apply the avatar characters discussed last time to your unconscious and have it speak for you. This would be a "self-translation machine" that retrieves information from what you might call your unknown self.

Feeling the future through synesthesia—the mysterious phenomenon that spans the five senses

Wouldn't it be good if the workplace had a tool that displayed the progress of projects with colors and shapes so that you could check and edit tasks at a glance? Departments that look as if they won't be able to meet the scheduled deadline might be displayed in red. Or maybe the departments that are the bottlenecks in the organization might have a thin shape and appear overstuffed. I'd absolutely love to have this at the anime studio, though if the bottleneck it found turned out to be me, it might be a bit awkward. (Laughs)

Another thing this sort of technology could be used for is entertainment. For instance, wouldn't it be great if there were a tool that enabled you to compose music as though you were drawing a picture? Suppose you could express the shape of the music graphically and compose it via sight. Innovative products that are similar to this idea, like the TENORI-ON (a Yamaha product), actually do exist. But it would be interesting if they made a giant leap from what we have now and we could invent technologies with which you could display texts graphically, or as three-dimensional objects, and edit them. The calm scenes might be round, and the exciting scenes might have a jagged shape.

There is a mysterious sense that is like this that cuts across the five senses—colors and language, or sounds and colors, getting crossed, and so forth. This phenomenon is known as synesthesia and is under active research. If we had tools that recreated this still relatively unexplored sense, wouldn't you want to make the most of them?

I myself have a sense that is something like synesthesia. When I would write a script—storytelling—I would project the sense of movement from when I animate into the process. I might have synesthesia between movement and storytelling. As another example of this sort of thing, surely you've heard that there are many people who have, at some time, gained inspiration from music.

I would be thrilled if Ricoh—a maker and a leader of our paper-based society—developed technologies that exploit this type of corporeal experience.