Multi-sensory Data Representstion: Going Beyond Data Visualisation





Human beings are imbibed with the quality to process complexities.The flow of unprecedented levels of complexities may be overwhelming but we are built to make sense out of it all. This capacity to process intricacies emerges from the unified sum of our senses.

Today, new types of digital mix are finding their way into our lives and while they are just as similar to to the data that our body processes, we usually interpret it through the sense of sight alone. Visualization dominates the world  of data representation, with good reason too. Visual tools of  simplification of data sets are extremely effective in identifying trends almost effortlessly.

With all the known advantages of visualization, there are some inherent disadvantages of dependence on a single sense for understanding purposes. For example: Despite the brain being wired for high capacity intakes and processing of images, this information in cached in a memory bank that decays quickly. Every second this part of visual memory is overwritten with new inputs. Whereas, our sense of smell is processed and kept in sync withthe long term memories. This is the reason why our earliest memories are attached to scents!

Because of diverse sensory experiences, addition of sound, taste, etc. to sight would widen the intensity of data experiences, with more impact than a one-mode representation.

 Emerging potential of multi-sensory data representation can be seen through the following examples:

Screaming Volcano :  Sounds of volcanoes just before their eruption are   being recorded by Scientists in Alaska. Volcanic activity is typically monitored through the subtle physical tremors of everyday seismic activity. There is a distinct sonic pattern that precedes a volcanic eruption. the signals have a tee-kettle like scream that takes place after a rhythmic drum-like build up. This acceleration and deceleration of the rhythm helps identify different activities leading up to the eruption, despite the researchers not being entirely sure of where the sounds originate.

Ghost Food : Ghost Food is a project consisting of a wearable device for emitting familiar, food-related scents, complemented by an odorless, “edible textural analogue” for simulation of the eating experience, by artists Miriam Simun and Miriam Songster. Even in the absence of food, by recreating the experiences of smell and chewing, a person’s mind creates the perception of flavor. Application of the mutually affected nature of taste, texture, and smell by mapping each sense to a distinct data point would generate a powerful multi-modal data experience unattainable by any other combination of senses.

Quotidian Record :  Brian House created a vinyl music album for audibly representing the places he visited over a year, using Geo-location data. Every location in a city is represented by a note in the musical scale while each city by a related musical key.On starting this record every revolution of the record recounts the locations visited that day, thus, working as a 24 hour clock. Patterns of behavior and movement begin to reveal, as the record spins and the sounds play. Work days, weekends, vacations, and holidays can all be distinguished from each other. On disassociating the locations from a map, we can hear patterns that may have been obscured by traditional visual representation.

 Scentography :  The relationship of emotion and smell with the Scentography device, an analog system designed to capture and reproduce odors is being explored by Artist Amy Radcliffe. Taking into consideration how tightly our emotions are linked to our olfactory senses, capturing and replicating the scent of objects and places might be enormously potent in creating impressive data experiences.

With the emergence of the extra-visual era of data representation, it is essential that we remember, the goal is not to find the best alternative or complement to visualization but to experience the data more richly. Human beings are evolved to experience data, inspite of this, we have only scratched the surface in comparison to our ability to understand data in purposeful ways. Looking beyond the conventional visual representation offers new opportunities to discover and communicate insights from data.




Previous
Next Post »

Popular Posts