Tag: synaesthesia

Do You Hear What I See?

Allen Otte's set up for Begin Again (photo courtesy Allen Otte)

I spent my youth playing notes on a page. And if you’re reading this, there’s a good chance you did too. This notation, particular for what we think of as Western music, is merely one graphic, albeit specific, representation of musical sound. And some of it is quite pleasingly arranged on the page, with calligraphy and shaped staves. But connections of music to visual art are as old as music notation itself.

Chant was notated with beautiful framing on the pages. Mussorgsky’s Pictures at an Exhibition translates the paintings of Richard Hartmann just as Debussy’s La Mer is a sonic response to Katsushika Hokusai’s Great Wave Off Kanagawa. William Grant Still took as his subject works by Richmond Barthé, Sargent Johnson, and Augusta Savage in his Suite for Violin and Piano. Gian Carlo Menotti broke through his writer’s block when he visited Botticelli’s Adoration of the Magi to come up with perennial holiday favorite Amahl and the Night Visitors, and Lady Gaga was likewise inspired by the same artist’s Birth of Venus for her own “Venus.”

These visual connections give the listener a starting point for understanding, which is especially useful in the field of experimental music. What is unidentifiable sonically can trigger a memory or a feeling when it’s attached to a visual. A visual that inspires the composer or improviser is sure to also inspire audiences to a fuller and more moving experience.

A visual that inspires the composer or improviser is sure to also inspire audiences to a fuller and more moving experience.

The Kentler International Drawing Center is driving this connection home with its now-touring exhibition Music as Image and Metaphor. The Kentler Flatfiles have been accessible to Brooklyn visitors for three decades, and curators planned to bring a selection of the collection to the Bartlett’s Center in Columbus, GA this past year. This would have combined with performances by composer/pianist Michael Kowalski and percussionist/composer Allen Otte via the music department at Columbus State University.

In a dilemma familiar to many last year, by October 2020 it was decided that the plans had to change. But Kowalski and Otte did not completely abandon the concert – they instead created a lasting musical installation, able to reach far more visitors than a single performance, with an opening in January 2021. For 40 pieces from the collection, Kowalski and Otte would create individual short musical responses. 40 new pieces of music, connected to visual works, accessible in the gallery and also online. A setup that allows the visitor to absorb themselves in the aesthetic conversation, or, exist within the infinity mirror of creativity.

Both Kowalski and Otte, as well as curators David Houston and Florence Neal, were happy with the result, and now the exhibition is headed to the Ohr-O’Keefe Museum in Biloxi, MS this month.

Allen Otte is a member of the Percussive Arts Society Hall of Fame. With the Black Earth and later Percussion Group Cincinnati he has been on the cutting edge of percussion-based chamber music. (Note: the author is a former student of Allen Otte.) Michael Kowalski was a pioneer of computer-based composition, who moved from chamber music to opera when he founded The Postindustrial Players. The two overlapped as students at Oberlin, and have collaborated before. But while being quite like-minded artistically, their approaches could best be described as opposites.

Knowing the likely answer, I asked both men if it was easier to write one 20-minute piece or 20 one-minute pieces.

Otte found the episodic nature delightful. “I could boom, you know, get an idea, make a response and not be responsible for actually much more than than the idea and the response. And in a minute or 90 seconds, it’s gone.” Percussion being an area where less is more in many cases likely made this more intuitive. “If it were twenty one minutes from me, I would have been uncomfortable,” he said. But he had expected Kowalski, who lists “composer” first among his occupations, to keep the game at a high level.

Kowalski agreed that the two are of a different mind, and thinks an attentive listener could take note of different kinds of craftsmanship happening. But that’s part of the fun, “because you don’t get in one person’s groove and stay there. It takes 45 or 50 minutes to actually hear the whole thing. If you just walk through the show and spend a minute on every piece, that’s how long it would take.”

Guests can take a tour through the exhibition, listening to pieces inspired by each piece of art. There is no stated theme, and no planned progression. The locations in Columbia and Biloxi are set up differently, with the images in a different order, so if a story can be extrapolated, it will be different than any other version of the exhibition. This includes an online visit, which can of course be in any order one likes.

In the compositional process, nearly opposite approaches were both successful.

Kowalski outlined specific procedures for himself, almost like a game:

Music as image:

  • Provide a soundtrack (as if the image is a film) or
  • Use the image as a graphic score

Or music as metaphor:

  • If the artist were making music, what would this image sound like? or
  • Enter a dialogue with the visual art

Random selection of these approaches created structure – more of a puzzle to solve and less of a blank page. And he applied these four procedures with a simple shuffle of the deck – mostly sticking to whatever process came up, no matter the image.

Otte was more intuitive, keeping a chart of the images he had an immediate reaction to, and curating himself from there: asking “whether I was doing too much of one kind of thing and whether I really ought to find a way to push myself to think about a piece in a different way.”

Both Otte and Kowalski spent time studying with composer Herbert Brün, who was a pioneer of graphic notation, and who is also represented as a visual artist in the Flatfiles. In Otte’s hands, responding to Brün’s piece was unexpectedly his most difficult assignment.

Three computer generated graphics by Herbert Brün

Three computer generated graphics by Herbert Brün–Orchestra Model One (1971), Ensemble Analogue Four (1974), and Web I (1971), image courtesy Kentler International Drawing Center.

“Herbert’s piece was one of the hardest ones to do and one of the last ones that I came up with,” he said. But also pointed out that throughout the project, difficulty often yielded a better result. This is possibly because some of the pictures presented a challenge, or because the challenge demanded more time be taken, and led to more self-questioning. Of Brün’s work he noted, “Well, actually, that’s the one that’s somewhat strong, that has some substance to it.”

Hear Otte’s response to Brün’s Orchestra Model One here.

For Kowalski, who is a white man, this challenge came in the form of an image of musicians at New York’s iconic Five Spot by biracial artist Robin Holder. His randomly selected procedure was to create a soundtrack – something that could easily have come across as an appropriation.

Five Spot 2 is one of the more literal images in the entire exhibition, so there was no way to ignore what was in it.

Five Spot 2 is one of the more literal images in the entire exhibition, so there was no way to ignore what was in it. “I had to be honest and embrace that. So that was a toughie.” So in this one case, he did break his procedural “rules,” writing what he felt was a more appropriate musical response. He also recruited an ensemble. Once again, having to think a little harder being a good impulse “that just forced me to come up with something else, maybe something better.”

Robin Holder's drawing of a group of jazz musicians playing instruments and singing.

Robin Holder: Five Spot 2, stencil monotype, 22″x30″ (2005), image courtesy Kentler International Drawing Center.

Hear Kowalski’s response to Five Spot 2 here.

While the ensemble was an enjoyable addition to the project, one standout is fairly minimal, as Otte responds to art by Mary Judge using just an amplified pencil.

The museum’s notes call the music “often surprising, sometimes baffling, always illuminating.” The connection between the 40 works chosen (out of 2000 options) by David Houston and Florence Neal is up to the beholder.  The same can be said about the pieces of music.

Otte felt a connection with the works by relating to what he called the performative aspect of an artist–the idea of still engaging an audience while the visual artist’s work remains still. Whereas Kowalski found a kinship with the act of creation – making a picture being analogous to making a sound. Different results, but the mindset implies a similar procedure.

All of which are ideas that can apply to other visuals when they combine with music–especially dance, where both Otte and Kowalski have a great deal of experience.

“I can only say that I’ve been, more often than not, astounded at what dancers are hearing in music and how they experience music and it’s often fascinating,” Otte said. In his experience dancers may give apologies for not “knowing” an appropriate musical term, while their assessment of the piece is generally quite insightful.

Kowalski also noted the complexity of choreography as a visual form: existing in three dimensions and moving. “If you’re sitting beyond about row 12, you’re seeing a great deal of usually very complicated forms, tracing patterns, on a fairly large stage.”

A previous collaboration between the two featured this interaction. Kowalski wrote a piece for the Percussion Group Cincinnati called Rebus, which includes choreography with flag signals. Initially composing a storyboard, once again the visual existed before the sounds. But, that piece was quite concrete – something Kowalski has always found essential working with dancers.

“Unlike musicians, dancers don’t notate, usually they don’t go into a rehearsal with a bunch of things in their head already,” he pointed out. “They work it out. It’s a very different way of working from most musicians that I know.”

There is no one right way to do a project like this. But like any collaboration, trust must be involved. If the creators are open and welcoming to each other’s vision, then brilliant combinations are possible. If we were to call the visual and the musical participants “sides” of the equation – the sides have to balance, and be somewhat open to the other’s contributions. Kowalski describes this as a tension, much like a conversation. But to be successful, each factor, visual and musical alike, must point to the other.

There is no one right way to do a project like this. But like any collaboration, trust must be involved.

“Some people dig it more visual, and then they get into the music and the other people the other way around, and I just think that’s ideal,” he explained. “I’m very happy about that.”

Despite their different approaches, both musicians planned and charted and graphed to create each of these responses. Otte describes the planning as a math problem. “The calculations that went into that final one minute; that final 60 seconds repeated for each of us 20 times in one way or another.” But also occasionally the minute of music came quickly and easily. “The ones which just came in in some burst of fun, we stuck with a few of those.”

Otte and Kowalski will be live at the Ohr-O’Keefe Museum for a talk and performance of even three more premieres. Forms of falling dust is a work for prepared yang-qin by Rachel C. Walker, a former student of Otte. Another collaboration between Otte and Kowalski called How To Compose Yourself involves a fairly frenzied piano part with percussive commentary. And the concert includes a new iteration of Begin Again, a work by Kowalski whose material stretches from the year 1597 to 1977 and now to 2021. In Begin Again a treatise by Thomas Morley was interpreted on an IBM computer by Ed Miller. A 1977 rendition included the voice of soprano Marlene Rosen, and this version it will include today’s additions from Otte and Kowalski.

The act of drawing on decades of material is part of what makes the project feel so substantial. Music originally created on an IBM the size of a linen closet, being watched and heard on a phone that fits in my hand, still feels fresh and new in this context. And while these pieces of music once again come to life thanks to fresh realizations, they also have renewed meaning thanks to the pairing with another artist’s visual material.

Music originally created on an IBM the size of a linen closet, being watched and heard on a phone that fits in my hand, still feels fresh and new in this context.

The clichés about art and music would tell us that the two aesthetic forms are bound to go together. I leaned into one of these, by Jean-Michel Basquiat, in my conversation with Otte and Kowalski.

“Art is how we decorate space. Music is how we decorate time.”

“Decoration,” said Otte. “That’s a loaded word.” Kowalski objected as well.

But at the surface level he immediately conceded that music could be “delightful if it is in fact decorative and entertaining.” And Kowalski identified “entertaining” as a secret word.

“That’s the word that overlaps: ‘decoration,’” Kowalski said. “Decoration is congenial and attractive and so is entertainment when it’s any good, I think. And so I would use the word ‘shape’ instead of ‘decorate.’”

So Basquiat is possibly correct, depending on what the music has to say. Whether or not you can welcome the word “decorate” for a serious piece of music is up to you, just as whether or not a piece of art “shapes” your space. And the fact that we’ve returned to these kinds of philosophical artistic conversations is another sign that we’re emerging from the harshest closure in the history of music with our thoughtfulness intact.

Development: musical image / Michael Kowalski's music sketches for "Untitled" by Kazuhiro Nishijima

Development: musical image / Michael Kowalski’s music sketches for “Untitled” by Kazuhiro Nishijima, images courtesy Kentler International Drawing Center.

As a pandemic-pivot, this project was enormously successful in that some music-making happened at all. While the music world navigates a bumpy road to a new normalcy, this project is quite possibly a model. Not just of the value of interdisciplinary connections, but also one of flexibility and access.

While the music world navigates a bumpy road to a new normalcy, this project is quite possibly a model.

Music as Image and Metaphor has visual and aural elements that are complete statements on their own. It can be experienced at an individual level, at one’s own pace. And it’s available in varying degrees of in-person participation, including online. And geographically, it has been available to viewers in the southeastern USA. While the Kentler Flatfiles reside in Brooklyn, they have been available in this form to viewers in Georgia and Mississippi. Modeling and sparking conversations – musical dialogues – that allow us to grow our audience, our depth as artists, and our own creativity.

40 Flatfiles down, 1,960 to go.


This exhibition of the Kentler Flatfiles includes pieces by the following visual artists: Herbert Brün, Beth Caspar, Phillip Chen, Abby Goldstein, Takuji Hamanaka, Keiko Hara, robin holder, Richard Howe, Hannah Israel, Mary Judge, Kazuhiro Nishijima, Ralph Kiggell, Rosalinda Kolb, Jiří Kornatovský, Robert Lansden, Simon Lewandowski, Jim Napierala, Florence Neal, Margaret Neill, Morgan O’Hara, Gahae Park, Jaanika Peerna, Scott Pfaffman, Orlando Richards, Susan Schwalb, Viviane Rombaldi Seppey, Molly Snyder-Fink, and Hugh Williams.

Understanding the Polychromatic System

We seem to be at an accelerated phase in our musical evolution, where isolated methods of music practice are rapidly multiplying without a framework of integration and orientation for musicians and listeners to grasp. The polychromatic system is one framework of integration for the various scale configurations of micro-pitch music.

Isolated methods of music practice are rapidly multiplying.

The polychromatic system is oriented toward exploring the outer limits of micro-pitch awareness and its expression in music, from a perceptual rather than conceptual (i.e. mathematical, theoretical, analytical) perspective.

The polychromatic system is based on principles of associative synesthesia: learned associations and conceptual/perceptual integration of audible pitch with visual color. Recent research has shown that associative synesthesia can be developed with practice. Music is an optimal area in which to extend the possibilities and potential of synesthetic awareness, both aesthetically and scientifically.


  1. To simplify and encourage musical curiosity—exploration and discovery of infinite             possible pitch scales and their sonic combinations (intervallic or other).
  2. To open new worlds of musical expression, experience, and composition.
  3. To aid in the development of new ways of ‘hearing’ sound/music and the world.

The standard 7-white, 5-black key layout of keyboard instruments.

Imagine a chromatic keyboard where each key is split. The front half of the key plays the conventional chromatic semitone pitch while the back half of the key plays the quartertone in between each front-key pitch (24 pitches per octave). We could distinguish these pitches clearly by assigning the quarter tones at the back of the key to a pitch-color (let’s say violet).

A keyboard layout for a polychromatic system of 24 equal divisions of the octave.

Moving up to 36 edo (equal divisions of the octave) brings new complexity. Now we can imagine a pitch-color above (say, blue) and a pitch-color below (say, red) each chromatic pitch. One way to describe them would be to say that, in the key of ‘C’, C-red is flat-ish and C-blue is sharp-ish relative to the chromatic pitch. The problem here is that the terminology of flat and sharp are embedded in the chromatic language both as a pitch definition (Db, C#) and as a pitch modifier (bb, x). By applying the concept of pitch-color, we can avoid both the confusion in terms and extreme notational complexity (countless, incompatible pitch-modifier symbols) ‘bolted-on’ to chromatic black and white notation.

A keyboard layout for a polychromatic system of 36 equal divisions of the octave.

This describes how I proceeded in trying to work with very high pitch-resolutions. The polychromatic system evolved to allow the creation of a simple notation and theoretical language for writing, learning, and memorizing micro-pitch music within a pitch division scheme of 106 and 72 edo.

A keyboard layout for a polychromatic system of 72 equal divisions of the octave.

The pitch-color concept is not absolutely defined, so the values (in Hertz) of C-red are different depending on the scale division method used. In this way, the language says remains generally consistent, while being adaptable to any conceivable pitch scale. Unique pitch definitions are defined at the beginning of each composition. This enables an efficiency of learning and possibility of using of multiple pitch scales (simultaneously or via ‘modulations’), all under a unifying and intuitive pitch-color concept.

The pitch-color concept is intuitive by way of an easy association of micro-pitch to the visual color spectrum—from infrared to ultraviolet. A simple association can be made with a continuum of flatness to sharpness for each reference pitch (chromatic or other) to the visual color sequence we already know from the experience of seeing rainbows and other light diffraction phenomena.

I use equal divisions of the octave as a method of pitch division because it is a rudimentary and self-explanatory element to begin from in my early explorations of auditory limits of pitched-sound differentiation. My approach is to use the highest possible edo scales within each new keyboard design.

While there is one option for a major 3rd in a chromatic system, the polychromatic system offers several pitch-color variations of that interval.

With regard to intervals, the polychromatic system uses relatively defined pitch-color definitions and is based on an idea of intervallic relativity. So, while there is one option for a major 3rd in a chromatic system, the polychromatic system offers several pitch-color variations of that interval. This intervallic flexibility has audibly compelling implications and effects when exploring variations of each component within increasingly complex harmony.

When listeners hear microtonal music as ‘out of tune’, this impression arises from a chromatic frame of reference. A new foundational frame of reference or perspective needs to be established in order to appreciate microtonal music on its own terms and not in comparison to the macro-pitches of the chromatic system. This foundation is what I have been seeking in developing a polychromatic system. In general, microtonal music can seem extremely abstract for unprepared minds and ears, especially without a new framework of understanding (for reference and comparison).

The polychromatic system builds upon the fundamental concepts of the chromatic system (note definitions, harmonic principles, and music theory) as a common point of reference – and departure. From this common framework, increasingly refined levels of micro-pitch discrimination can be explored within a known system of musical understanding. As greater refinements of pitch and harmony recognition are developed, increased awareness can enable the recognition of further pitch-colors, as entities in themselves, without trying to force them into the conventional frame of reference: a coarse pitch-resolution, chromatic system.

In analogy, imagine the chromatic system as an old monochrome (black & white) dot matrix printer, with its chunky, quantized images. If you input a high-resolution image into that framework, it is ‘processed’ into a chunky, quantized image. The foundational framework of the coarse resolution, dot matrix system design must be addressed first.

A Low resolution dot-matrix image showing very pixellated letters as well as a very pixellated photo close-up of an eye of an animal

Paradoxically, ‘tradition’ has vital importance in the creation of new musical systems. Here, we are talking about tradition as our accumulated experience of the past, a shared frame of reference, an implicit basis and context of listening to and composing music—and not in the sense of calcified conventions of the past and present.

Paradoxically, ‘tradition’ has vital importance in the creation of new musical systems.

Polychromaticism is an approach and practice which uses chromatic music theory as an initial basis for conceptual anchors. In this way, extensions of established tonal and harmonic principles assist in the understanding of the new possibilities and implications of the polychromatic system. This creates a conceptual and perceptual bridge between our current and many future systems of music.

The use of chromatic conceptual anchors comes into play when I am trying to memorize a piece for performance. Using the conceptual anchor of a dominant 11th chord, I remember the constituent pitch-colors of the chord as minor modifiers within this larger conceptual structure, instead of having to remember each pitch-color individually.

My composing process is much more intuitive than analytical. As a result, I have neglected doing harmonic analyses, after the fact, to determine possible new theoretical models. My hope is that others with a passion for analysis will create new music theory models to aid in teaching the polychromatic system more efficiently. Philosophically, I see this as part of a larger 21st-century process which is based on collaborative innovation in music, science, technology, and the sound arts.

As a musician, it seemed impossible to gain a fundamental grasp of microtonal music because it consisted of so many discrete pitch-scale methods, all separately existing without a fundamental, underlying system.

With a polychromatic basis in notation and generalized pitch-color concept, any micro-pitch language could be quickly assimilated without having to relearn scale-specific notations. The current expansion of esoteric and fragmented micro-pitch scale methods, of reinventing-the-wheel (in notation) with each, can make wide exploration within microtonal music intimidating and difficult.

The use of non-chromatic color schemes:

A diagram of various hex-color schemes.

While the color schemes here are different, my impression is that these are examples of a modally optimized layout – in the sense that adjacent keys are 3rd, 4th, 5th, and 6th intervals rather than chromatic minor 2nd’s (or smaller) intervals. In the polychromatic system there are multiple pitch-color variations of each interval, rather than the single fixed-interval value in these layouts. Why be limited to just one major 3rd interval, when multiple intervallic pitch-color choices—singularly or simultaneously—could be available?

Ultimately, the polychromatic system exists to make the process of micro-pitch exploration and creation easier. And just as the polychromatic system has evolved from the chromatic system, it too will eventually become a (legacy) reference and conceptual point of departure for many increasingly sophisticated (non-chromatic) music systems of the future. The polychromatic system is one way of making the musical evolution toward triple-digit, pitch-scale resolutions easier to understand and create with.

As a musician and a listener, I experience music as a dynamic and evolving process, a creative interaction that we choose to engage in. Ultimately, the meaning and value of music comes from the quality and depth of this creative interaction.

Art doesn’t come to us, we must come to art.

The possibilities of growth and awareness gained through our engagement with art remind me of the idea that art doesn’t come to us, we must come to art. This idea expresses both a necessary receptivity to new perspectives as well as an active personal involvement which engages the listener’s creativity and imagination, in a similar way to that of the composer. In this process, the listener becomes a receptive-artist interacting with the compositions of the expressive-artist (composer). If we become what we practice, then exposure to, and interaction with challenging art can help us expand our integrated perpetual/conceptual awareness, and expose us to new dimensions of emotion and insight beyond the limits of spoken language.

In the next article, I will describe technical (physical technique and technology) aspects of my approach to polychromatic music, as well as some discoveries and implications that I have become aware of in my early explorations.

Complicity and the Chemical Senses

Blindfolded patrons sit around a table and eat various food items, smell perfume, and listen to music.

Jozef Youssef presents a multisensory meal for the launch of a new Hugo Boss fragrance. (Photo courtesy Kitchen Theory)

I spent the weekend composing two new pieces of music for another collaboration with chef Jozef Youssef of Kitchen Theory. This time around I was tasked with bringing out the sweet and sour elements of a lavender and pomegranate-based tapioca dessert, part of a multisensory meal that Youssef was organizing for the launch of a new Hugo Boss fragrance. (Previously, I composed music to bring out the sweet and bitter notes in Chivas Regal 18 Ultimate Cask Collection blended Scotch whisky for a similar event.)

My sweet texture is pretty similar to the sweet texture I wrote for the whisky pairing: consonant intervals and mellow timbres in a relatively high register, organized in short, ascending phrases, with a sprinkling of wind chimes.

The sour texture, while similarly drawing on the psychological literature of Charles Spence’s Crossmodal Research Lab at Oxford University, is a bit looser in its interpretation of the research. I used bright, buzzy timbres and descending phrases for contrast, and I picked up on an idea from the music I wrote for a 2012 Azul y Garanza Garciano to depict tartness with dissonance, not only in the musical intervals, but also in the slight detuning of my subtly shifting delay effect. To give me a wider continuum of intervals from which to choose, I composed my music in Harry Partch’s 43-note per octave scale, which allowed me to incorporate his notions of comparative consonance (the simpler the ratio, the more consonant the interval, which is the best explanation of consonance vs. dissonance I’ve encountered and serves as a finely graduated yardstick for crossmodal inquiry). I interpreted the faster tempi that have been linked to sourness in psychological studies more generally as high rhythmic density, reflected in my texture’s pulsating elements. Sourness feels higher than sweetness to me, so this piece is in a slightly higher register as well.

While I wasn’t specifically composing music to accompany a scent, scent plays a vital role in dining, and I think the challenges to accompanying scent with music are similar to taste.

Taste and smell are our chemical senses. They are what allow us to interrogate the makeup of our environment, providing information that perhaps might lead us in the direction of dinner or cause us to spit out something poisonous. The visual arts and the sonic arts arrive to us from a distance, via electromagnetic radiation or fluctuations in air pressure, but taste and smell require direct contact.

Philosophers have long debated whether the fundamentally different nature of these chemical senses precludes the elaboration of an art of ideas based on them, something that goes beyond the ancient and sophisticated traditions of perfumery or cuisine. Taste, which requires a visible delivery mechanism, perhaps has a leg up on smell, which is invisible (like music).

A few years ago, Chandler Burr lobbied New York’s Museum of Arts and Design to establish a new Department of Olfactory Art and was subsequently brought on as its head. His brief tenure culminated in the 2012 exhibition The Art of Scent 1889-2012. A man of passionate pronouncements, he comes right out and says of perfume, “It’s art.”

I’m happy to accept, along with Marcel Duchamp, that if someone proclaims his or her work as art, it’s art. However I’m not sure whether, during the long history of perfume-making, most perfumers have actually been making that claim. We’re not talking about the gradual acceptance of a new technology; unlike photography or video games (my primary arena of activity for the past twenty years, which only recently emerged from its own “is it art?” debate), perfume has been around for a very long time. It might take a little more work than the wholesale rebranding of an existing discipline to bring perfume and art into the same conversation. Perhaps we can repurpose Wittgenstein’s dictum that not every building is architecture to suggest that not every perfume is a work of olfactory art.

(Incidentally, while I’ve seen a couple of announcements in recent years about companies bringing the sense of smell into gaming, my very first game, the maybe not so great Leisure Suit Larry 7: Love for Sail!, may well be the first, way back in 1996. The game shipped with a scratch and sniff card, and at key junctures in the game, the “Cybersniff 2000” logo would flash and a sound would play (my processed voice), informing the player which of the nine squares to scratch, neatly addressing the question of synchronization. The soundtrack also featured a 12-tone faux-jazz composition, the sure mark of a young composer fresh out of music school.)

One of the considerations is practicality. In an interview with The New York Times, Holly Hotchner, the former director of the Museum of Art and Design in New York City who brought Chandler Burr on board, conceded that “perfume by its nature has to be wearable, which is not true of other art forms.” Paul A. Young, a London-based chocolatier who devised a charcoal-flavored chocolate to accompany a Francis Bacon painting at the Tate Sensorium exhibition acknowledges the assumption that food should be pleasurable when he says, “Some people will find it repulsive and not want to eat it, some will find it engaging and some people will love it.”

Music has always been considered an art form, but in the last few decades, sound art has emerged and evolved as a parallel practice distinct from music (although I think sound artists ignore the history of music at their peril). There’s no quantifiable difference between sound art and music. The materials are exactly the same; it’s a question of focus. With all of the current interest surrounding food and multisensory perception, I’ve been thinking for a while that taste and smell may be next in line to be appropriated by the art world. But this question of focus highlights the challenges that can arise when people from different creative disciplines come together (as I described in an earlier essay).

As an example, when I attended “The Architecture of Taste,” Pierre Hermé’s presentation at the Harvard Graduate School of Design a few years ago, it was fascinating to get a glimpse into a great chef’s creative process, and the pastry samples were amazing, but it was clear, especially during the Q&A afterwards, that he was thinking of his work in very different terms from the concerns of art and design grad students. Another leading pastry chef, Jordi Roca of El Celler de Can Roca, discussed his relationship to art in the film that documents the multimedia meal El Somni, devised alongside the other two Roca brothers, concluding simply, “I’m just a pastry chef.”

An assortment of desserts on a tray.

Desserts served at Pierre Hermé’s Harvard lecture included an “infringement citron” and other desserts made from unconventional ingredients such as wasabi. (Photo by Ben Houge.)

Dean Dutton provides a good overview of the issues surrounding the notion of olfactory art in his book The Art Instinct, and as I’ve been doing projects that combine food and music, it’s been useful for me to consider these arguments from a musical perspective.

One issue that he dismisses a bit too quickly, I think, is the idea of “internal relations.” He borrows this term from philosopher Monroe Beardsley, pointing out that, while we may recognize corresponding proportional relationships in the spatial layouts of a painting or the intervals of a musical chord, chemical compounds don’t lend themselves to this kind of perceptual organization that can serve as the basis for large, sophisticated structures. There’s no equivalent to an octave or a major 7 sharp 11 chord or a retrograde inversion that would allow a composer like Bach to spin out a ten-minute fugue from a subject of just a few notes. The wine aroma wheel is nowhere near as precise as a color wheel (can you rotate a raspberry note 180 degrees to obtain its opposite?) or the circle of fifths (what’s the secondary dominant of leather?).

A second obstacle is a question of the delivery mechanism, and here I’m thinking about a time-based sequencing or counterpoint of multiple scents that goes beyond the natural diffusion of a scent in the air once it is sprayed or applied. (This is more of an issue for scent than taste.) To think of scent in musical, time-based terms, it’s a challenge to consistently deliver a scent and take it away, allowing for temporal contrast. This issue thwarted some of the historical efforts to bring scent into cinema, such as Smell-O-Vision from 1959.

Many approaches to scent diffusion have been developed with varying levels of technological overhead. The Museum of Art and Design’s Art of Scent exhibition took place in a spare, white room, designed by Diller Scofidio + Renfro, laudably devoid of packaging and other marketing detritus. As visitors inclined their heads over small indentations in the wall, a puff of scent emerged from a sophisticated mechanism designed by Scent Communication in Germany. At Green Aria: A Scentopera, a collaboration between entrepreneur/writer/director Stewart Matthew, perfumer Christophe Laudamiel, and composers Nico Muhly and Valgeir Sigurdsson which premiered at the Guggenheim in New York City in 2009 (about which NewMusicBox’s Frank J. Oteri has written extensively), scent was delivered to each spectator via a “scent microphone” (functionally more like a “scent speaker”) at each seat, which could be positioned as close to the nose as desired. The inaugural installation of the recently opened Museum of Food and Drink in Brooklyn, masterminded by cocktail impresario Dave Arnold, features a “Smell Synth” that allows visitors to experiment with different odor compound combinations at the push of a button. This echoes the “Olfactiano,” a “scent piano” developed by Peter De Cupere, which he has used for time-based scent-oriented performances such as his Scentsonata for Brussels presented at the Cordoba festival in 2004. At the Crossmodalist rehearsal I attended in London last April, the technique was low-tech but effective; I was blindfolded, and different perfumes (designed by Nadjib Achaibou to evoke concepts such as “lust” and “sorrow”) were manually wafted in my direction, sometimes simultaneously, by unseen hands at different moments of Chris Lloyd’s live performance of Liszt’s piano transcription of Wagner’s “Liebestod.”

A third consideration is representation. I’ve written in the past about what I called the “abstract” quality that food and sound share. By this I meant that, while music can be linked to an external narrative via text or theatrical context, coherence is essentially the result of internal relations, to return to Monroe Beardsley’s term, i.e., a note has meaning in the context of a chord or scale or harmonic structure and how these structures evolve in time. Even without recourse to the same kinds of internal relations as music, tastes and scents are constructed and developed by combining chemical compounds. This is in contrast to the visual and literary arts like painting, sculpture, choreography, cinema, literature, poetry, and video games, in which meaning-making via external representation is the norm. I think it’s important to consider that an extracted chemical essence of an object is still a form or an attribute of the object; the scent of a rose derived from a rose cannot be said to represent a rose in the way that a painting represents a rose, because in a real sense that essence actually is the rose. Representation is the use of one thing to depict another thing; as Magritte put it, “Ceci n’est pas une pipe,” and no pipe extracts are required to produce a painting of a pipe.

My usage is different from the way the term is used, for example, in Annick Le Guérer’s article “Olfaction and Cognition: a Philosophical and Psychoanalytic View” in the weighty 2002 compendium Olfaction, Taste, and Cognition. Here, the historical assertion (shared by numerous philosophers) is that scent is not “capable of abstraction,” by which she means that a common attribute cannot be extracted from multiple scents in the way that we might refer to an abstract color. This is reflected in the lack of vocabulary for describing scents (at least in English); we generally simply describe scents according to their sources or, in the case of synthetic compounds, by analogy.

So perhaps my earlier usage of the term is nonstandard, but I think the ideas are related: just as mimetic music was by far the exception prior to the advent of recording technology (with timpani evoking thunder in Berlioz’s Symphonie Fantastique serving as an example of a rare exception), it’s difficult for a taste or a smell to be “about” anything other than itself.

I’ve heard people assert that what music represents is emotion, and I could imagine this argument being applied to smell or taste, but evoking and representing are not the same thing. As Frank J. Oteri pointed out in our recent correspondence on the subject, “Associations (e.g. major = happy, minor = sad) are the result of acculturation and not universal. The same, though, could be said for scent.” He notes the curious coincidence that, as I mentioned above, music and scent are completely invisible.

I’ve also heard some people conflate tonality with representation, and there certainly are clear parallels between figurative painting in the visual arts and tonal structures in music, but tonality strictly speaking still does not represent anything outside of music’s “internal relations,” and in fact it is these relations that give tonality meaning. Nonetheless it is certainly worth observing that abstraction in painting and tonality in music experienced parallel ruptures in the tumultuous early 20th century, and it is surely a result of the same cultural forces that Chanel No. 5 was introduced in 1921, unprecedented in its lack of reference to natural scents, “a perfume like nothing else,” in the words of Coco Chanel.

At a certain point, while pondering these questions, I realized that music has been here before. Like many undergraduate music students, I read about the absolute vs. program music debate in my music history classes, and (like a lot of students, I suspect) I was left wondering what the big deal was. On one side of this virulent debate you had Richard Wagner espousing opera as a total art form driven by narrative and deriving power from the coupling of art forms, while on the other side Eduard Hanslick argued for purely instrumental music as the art form’s pinnacle of expression. I recently picked up Mark Evan Bonds’s excellent new book on the subject and realized just how far back this question goes. He organizes his argument around the ideas of essence (what music is) and effect (what it does). This historical perspective has the potential to serve as a useful framing device for discussions of an art based on the chemical senses.

There is a category of “olfactory artists” that has emerged as a distinct practice from that of perfumers, and it seems to be expanding rapidly. Many of the artists working with scent are not working only with scent, and often they’re not working alone. Some of these artists are crafting the scents themselves, but oftentimes the chemical component is outsourced. Pamela Rosenkranz worked with perfumers Dominique Ropion and Frédéric Malle and sound designer Emar Vegt on Our Product, an installation at this year’s Venice Biennale that presented a huge pool of liquid the color of an averaged European.skin tone. Belgian artist Peter De Cupere, perhaps the best known artist in this arena, has been working with scent since the 90s, often incorporating smells into large-scale mixed media sculptures, as in The Smell of a Stranger, presented at the Havana Biennial, which engineers indigenous Cuban plants to give off scents redolent of Western culture. Brian Goeltzenleuchter has been working on a project to capture the smell of each neighborhood in Los Angeles.

Arnica Yi’s 6,070,430k of Digital Spit, presented last summer at MIT’s List Visual Arts Center, bridged the worlds of scent and taste by building an immersive installation inspired by the “Mint Pond” dish at Ferran Adrià’s elBulli restaurant. She notes, “I think the most radical artistic statements are being made in the world of cuisine. That interest translates and seeps into my approach to smell. Even though I don’t work with food, I feel the sensibilities are shared.”

Soundscape for Pamela Rosenkranz – Our Product. Swiss Pavilion at Biennale Venice 2015 from Emar Vegt on Vimeo.
The high degree of collaboration that many artists are already exploring points one way forward for an art of the chemical senses. Maybe the lack of “internal relations” and problems of diffusion preclude certain types of standalone olfactory expressions, but by linking art forms, sophisticated new kinds of experiences are possible. Let music aid with the time-based elements. It’s not unlike setting a text or scoring a film; another art form defines the structure. Ramón Perisé of Mugaritz writes that, “how cooking can participate in a multisensory spectacle is something that, in my opinion, is in the first phases of exploration.” The idea of putting different art forms together into a multimedia event is the reason I chose the term “food opera” for my project in the first place.

Listening to music is something we do together. So is dining. As I’ve been working on my food opera project, I’ve spent a lot of time thinking about the overlap between these two spaces: the restaurant and the concert hall. Food represents community, maybe literally, in the same sense of the word I used when writing about “representation” earlier. When we think of fellowship, we often speak of breaking bread together. Dining is one mode of being in the world together.

To orchestrate a communal dining experience as an artwork is the kind of tactic associated with the relational aesthetics movement of the 90’s. Rirkrit Tiravanija’s 1992 piece Untitled (Free), now in the collection of New York’s Museum of Modern Art, involved distributing free curry and rice to visitors in a gallery space, and since then, Tiravanija has presented a number of other pieces that involve people coming together to eat. Together with his longtime gallerist, Gavin Brown (a collaborator on the original exhibition of Untitled [Free]), he has been developing a commercial kitchen in Hancock, NY, called Unclebrother. In a recent interview, he highlighted some of the same community-oriented values I emphasize in my work, “It’s about eating from your surroundings.”

This is a big part of what my food opera project is about. I have observed that these events promote community in a unique way. Because each seat in a restaurant has its own speaker and functions as a source of music, diners become more aware of the people around them and in fact depend on the presence of other diners to complete their experience. The music foregrounded at each diner’s table becomes the accompaniment for the person at the adjacent table, such that everyone in the restaurant is involved, through their choice of dish and the rhythms of their meal, in facilitating the overall musical experience. There’s a lateral exchange of information that is unlike a typical concert; sound passes not just from performer to audience, but from diner to diner. This strikes me as a rich yet underexplored model for musical communication (something I’ve also explored in my piece The Tomb of the Grammarian Lysias, a setting of a Greek poem by Constantine P. Cavafy for unamplified voice and audience mobile devices).

At several events, we’ve gone so far as to incorporate field recordings from the farms that provide the ingredients and interviews with farmers into the soundscape, since Jason Bond, the chef with whom I’ve collaborated on these projects, is a passionate supporter of local, sustainable agriculture. We used sound to literally bring farm to table. My hope is that diners leave the restaurant with an increased sense of their interdependent roles in a larger ecosystem, which may encourage more responsible food consumption choices.

When I was attempting, in my poor Spanish, to describe this aspect of my work to chef Dani Lasa at Mugaritz, he understood immediately and responded with the perfect word: “complicidad.” Complicity. I asked if Mugaritz had ever done a dish involving sound, not in their various multimedia collaborations, but actually in the dining room. Dani told me about a dish called Mortar Soup with Spices, Seeds, Fish Broth, and Fresh Herbs. No matter at what stage people were in their meals, the entire restaurant was served the dish at the same time, and as each diner applied the pestle to the mortar, the whole room rang with a sound like Tibetan singing bowls. This experience was enabled by the restaurant, but enacted by each individual diner; everyone was complicit in the resulting sound.

A mortar, a pestle, and food.

Sopa de mortero con especias, semillas, caldo de pescados y hierbas frescas. (Photo courtesy Mugaritz.)

This is the goal of my food operas: to bring things together, connecting creative disciplines, connecting farms to restaurants, connecting people to their environments and to each other.

I hesitate to add this coda, but I feel this is an important point. I’ve been writing this essay in the wake of the terrorist attacks in Paris, and in Beirut before that, and of course you can keep going as far back as you like. In these factious times, more than ever we need to find ways to understand those whom we perceive as being different from ourselves as part of the same ecosystem, complicit in the well being of the world. As someone who’s lived a third of his life outside of his home country, I believe that cultivating empathy for others via direct interaction is the best hope for peace. One of the best ways to do this is to come together over a meal. Just as our chemical senses—Le Guérer uses the term “proximity senses”—require contact to perceive our surroundings, we need situations that allow us to come together and better understand our neighbors on this planet. When we break bread together, a meal can do that.

Vinfonies, Nessun Dorma, and Gastromorphology

A bottle of wine and a wine glass on a tale alongside an electronic keyboard, music notation paper, headphones, and a computer screen.

(Photo by Ben Houge)

At just about every technology-oriented conference I go to, I bump into some representative of the Music Technology Group of the Universitat Pompeu Fabra in Barcelona. Probably best known for developing the Reactable, the interactive music control system prominently used by Björk, the MTG is one of the most well regarded centers for music technology in Europe. I believe it was at the Game Developers Conference in San Francisco back in 2012 where I first met the MTG’s Jordi Janer, who has been conducting fascinating research into voice analysis and audio source separation, work that he has made available commercially via his side company, Voctro Labs.

Jordi has another side project as one of the founders of Vinfonies, a series of events pairing wine and music that goes back to 2009. Jordi himself has done a number of sound installations related to various aspects of viticulture, developing pieces based on resonating cava bottles, an interactive grape press, and the sonic byproducts of fermentation. With my food opera events I’ve sought to bring the sounds of the farm into an urban restaurant, but Vinfonies goes the other direction, using the festival as a way to bring wine-themed new media art to rural settings. The events happen annually at harvest time in Vilafranca del Penedès (part of the well-regarded Penedès winemaking region, one of Spain’s sixty-five or so regions that have earned the Denominación de Origen appellation) about 50km from Barcelona. The festival includes concerts, sound installations, and wine tasting sessions. Similar to the work of Spanish sound artist Francisco López, participants experience the wine and music pairings blindfolded, eliminating visual distractions to focus on taste, smell, touch, and sound.

While I was living in Spain, I had a chance to reconnect with Jordi at Music Hack Day Barcelona, sponsored by the MTG as part of the huge Sónar electronic music festival. (Check out my brainwave sonification hack from 2014.) The MTG partnered with Berklee to bring computer music pioneer John Chowning to Spain for a series of events last summer (including his first visit to Valencia since he docked there in 1953, back when he was a drummer in the U.S. Navy), and in the planning conversations, Jordi and I realized we shared an interest in music and gastronomy. He invited me to write a piece for the 2015 edition of Vinfonies, which took place last September. Over the summer, he mailed me a bottle of Azul y Garanza 2012 Garciano (the name, which might perplex some oenophiles, derives from its composition: 50% Garnacha, 50% Graciano), and I set to work composing my musical accompaniment.

BTW, a playlist of previous Vinfonies compositions (including one by Edwin Van Der Heide, whose fascinating sound installation “Spectral Diffractions” took over Mies van der Rohe’s Barcelona Pavilion as part of Sónar in 2014) is available on SoundCloud.

Composing music to accompany a taste-based experience presents some unique challenges, particularly when it comes to synchronization. One of the epiphanies that sparked my food opera project was the recognition of a meal as a time-based art form. It’s easy to recognize several time scales to a meal, from the succession of courses (even simply saving dessert for last) to the entropy that occurs as a hot dish cools or a frozen dish melts to the succession of individual bites.

Recognizing these time scales is straightforward, but synchronizing music to them is a much trickier proposition. Typically, restaurants don’t even try, and in this they resemble far too many video games that simply loop the same piece of music over and over or shuffle a playlist: the only point of coordination between the music and the restaurant is that while you’re in this space, this is the music that’s playing. (See the paper I presented at Invisible Places Sounding Cities in Portugal for a more detailed discussion.) A few restaurants that explicitly stress the multisensory angle go further: Ultraviolet or the various Kitchen Theory dinners will present a track of music synchronized with each course, but for this to work, everyone has to be eating at the same time. This approach—when the dish is served, someone presses play on the CD player—works best when there are a greater number of relatively small dishes that don’t take too long to eat.

To coordinate music with individual bites is still trickier. One clever solution is the Tasteful Turntable, devised by Lars and Nikolaj Kynde to synchronize several small bites with key moments in a soundtrack presented to four diners at a time. Another, somewhat less practical approach is Naoya Koizumi’s Chewing Jockey, which uses a photoreflector sensor and a bone conduction speaker to alter the sound of chewing. But when the dining experience is opened up to multiple diners eating asynchronously (i.e., the way the usually eat), the logistics of synchronization become much harder to address. One solution is to give diners headphones, as Heston Blumenthal does in his famous Sound of the Sea dish, or in performance artist Marina Abramović’s collaboration with chef Kevin Lasko, Volcano Flambé. Headphones works as a personal experience, potentially a profound one, but they also cut people off from the environment and those around them. The other solution is to put a speaker at each seat, broadcasting sound unfettered into the restaurant space, and then (this is the hard part) avoiding cacophony by finding a way to incorporate the bleed of music from adjacent tables into the experience, so that all the sound in the restaurant is coordinated. This is what I’ve done in my food operas, and from my observation, this is the only solution that scales to an evening-length experience.

This solution requires all of the musical compositions written for a food opera to be modular, so that they can be put together in any combination and still sound harmonious. At the events I organized with chef Jason Bond in his restaurant Bondir, there were five courses, and diners chose between two possible dishes per course, so that’s up to ten different pieces of music that could be playing at the same time, each at different points in the composition, from the twenty-six seats in the restaurant. I wrote custom software to generate a new version of each musical texture and play it on the correct speaker every time a diner was served the corresponding dish; because the music was generated on the fly, the software could also ensure that all of these individual streams of music were coordinated in harmony and rhythm. In order to make this coordination as apparent as possible, I limited myself to a diatonic scale (slowly transposing over the course of the evening, with all of the music in the restaurant programmed to conform to the current key) and a steady underlying beat referential of 188 beats per minute.

However, with this Vinfonies commission as a stand-alone piece, I had no such constraints, and so I made it an objective to make the most of my liberty. I also wanted to avoid some of the techniques I had used in the past that I decided were a bit “too easy,” so I decided to eschew steady drones, for example, and the overt diatonicism and rhythmic grid of my previous work.

When I’m developing music to accompany some gustatory stimulus, my process isn’t so different from how I’d approach a video game or a choreography: I sample the experience of the thing I’m scoring (or at least a description or concept art or whatever state of completion the thing is in), evaluate my response, and try to capture what I hear in my mind’s ear. In this, the composer has a great advantage over the researcher; whereas academic studies might try to compare the appropriateness of different types of wine to existing music (as in a recent study contrasting Debussy with Rachmaninoff), a composer can hone in on and express an ideal imagined sound without having to choose between existing examples. (This may lead to less quantifiable data, but in some ways it resembles another study, in which participants were asked to find the single note on a keyboard that best corresponded to a scent stimulus.)

An interesting point came up in Oxford last February, when I collaborated with researcher and wine expert Janice Wang in an event at Alistair Cooper’s 1855 Oxford Wine Bar. We presented three pairs of wines alongside three pairs of musical textures and asked participants to determine which musical texture best matched each wine. In the ensuing conversation, it became clear that some people chose music to evoke the wine, whereas others chose music to complement the wine. This may be frustrating to the researcher, but it’s exciting to the composer, at it shows how much room there is for creativity in devising music/food pairings. Music really can serve as another kind of seasoning, flexible in the same way that an ingredient like mint might be equally at home in a sweet or savory dish. We can recognize this phenomenon from the world of film scoring; it’s an exercise we even give to students at Berklee, to compose different soundtracks to the same film clip to give it a different emotional spin, to make it happy or sad, wistful, nostalgic, or ominous.

The Azul y Garanza 2012 Garciano provided a great opportunity to stretch out a bit. It’s a complex wine, and working at the intersection of food and music, I’m sometimes bemused by the fact that, while it’s common for people to decry complexity in music, complexity in wine is pretty universally considered a positive attribute. Jordi Janer shares an interesting observation on this point; he tells me that when pairing music and wine at the Vinfonies events, he’s found participants much more receptive to challenging, experimental soundtracks than they might ordinarily be. Unlike the whisky project I wrote about earlier, my objective here was not to translate aspects of the wine into sound so much as to create a sonic context for it, and in this case I envisioned a kind of mysterious, mystical setting.

I decided to stick mostly to an octatonic scale, allowing the music to float for long stretches without reinforcing a particular key. I sought a harmonic language that was more dissonant than what I had used in the past. The spiky, disjunct marimba patterns evoke a bit of the spicy quality of the wine. The music has no pulse, evoking a kind of timelessness. I wanted a density to the music that reflected the wine’s complexity, and the delay lines help achieve that, providing a sense of depth; I imagine myself peering into the wine as though it were a dense forest. I tend to prefer delay lines to reverberation in these kinds of pieces, and I think there’s also something liquid in the rolling echoes. The lengths of my delays are periodically changing, which results in a bit of pitch shifting during the transition, and this subtle detuning evokes a kind of tartness. (I’m not sure if this kind of detuning has been subjected to a formal study, but I wonder if there isn’t a link to the expression “a sour note.”) Periodically, as a kind of refrain, a jutting theme outlines an augmented chord, stretching out over a tenth, and alters the mode, swapping a D sharp for a D natural, serving as a kind of punctuation. At these moments, a sustained bed based on cello sounds makes a rare excursion into bass clef territory. In some of my previous work, low cello phrases have been successfully linked to tannins in wine, so I decided to include cello here for the same reason. But in general, the frequency range of the music is in a high register, which reflects the sweet and sour fruit components I notice in the wine. Musical ideas float along as objects for the listener’s consideration, not developed, but juxtaposed for the listener to parse and contemplate. Overall, the music is peaceful, emphasizing stability and resolution, reflecting this well-balanced wine.

(If this description is a bit tedious, note that one of my original reasons for pairing music with food was the recognition of how poor words are for describing music and food, suspecting that they might do a better job of describing each other.)

The instrumentation comes from manipulated recordings of myself (whistling and playing piano strings with mallets) and my Berklee Valencia colleagues Victor Mendoza (marimba master) and Sergio Martínez (percussion savant), with Berklee alum Ro Rowan’s cello recurring periodically.

In the above description, you may readily detect the influence of a composer whose music has deeply influenced my thinking about real-time musical systems and video games—Olivier Messiaen. Much of his music exists in a continuous present, which is exactly what game music must do, as it waits for the next event to signal a transition. By making this piece almost a kind of homage, I sought to link my work to an ongoing musical tradition, something I think is important when venturing into new territory, especially in light of some of the cultural crosstalk I mentioned in my previous post.

My compositional process was idiosyncratic. First I determined the scale and the types of sonorities and simultaneities I wanted to use, and I sketched these out on staff paper, along with some thematic ideas. Then I took some of the instrumental recordings I had prepared to use as source material and turned them into playable instruments in Max/MSP. I wrote a program that would take any note I played on the keyboard and build one of my previously determined chords, in any approved inversion, on top of it; in this way I generated the “mallet piano” part, allowing me to manually control the range, rhythm, and dynamics of that part, while letting the computer generate the chords. Then I wrote a program that would look at the notes of the most recent two chords played by the mallet piano and choose from among them to generate a melody to play on the marimba; whenever I pushed a button, a new melody would be generated according to parameters I specified, and a different button allowed me to repeat a phrase with subtle rhythmic variations. The whistle part, except for the recurring theme (which was played manually), was also generated by looking at the most recent mallet piano chords and choosing a note from among them, although in this case I was determining the octave transposition, rhythm, volume, and duration manually by playing the keyboard. The rattle and cello parts were performed or input manually. To sequence everything into the requested time frame, I used Ableton Live, embedding my Max patches as instruments using Max for Live.

But wait: for this Vinfonies commission, since a linear recording of about three minutes’ duration was what was requested, I could have composed and sequenced everything linearly. So why this weird process involving programmed behaviors?

It comes back to the idea of synchronization. In most cases, eating and drinking are activities that can continue for an indeterminate duration, so I’ve been very interested in composing music that can continue indefinitely. (This is the same problem we face in video games, and much of my creative effort since starting in the game industry in 1996 has been addressed to it.) It’s possible that someday I might want to use this music again and present it in a real-time, indeterminate duration form. If I do, having these algorithmic processes already in place will make it easy to adapt; by reducing the input into the system (playing single notes or pushing buttons instead of playing full chords or phrases), I make it easier to replace my manual input with some automated mechanism. I also wanted to fit this piece into my growing body of “food opera” work, using instruments that I’ve built from acoustic samples in Max as my orchestra, organized around generative or procedural processes to accommodate the constraints of modular, real-time deployment discussed earlier. And I have to admit, composing algorithmic music in this way is simply an interesting challenge. So, since much of my work hovers around this idea, it just made sense to apply the techniques on my current workbench to this new commission. But more than any of these considerations, I feel that this approach to composing—using algorithms to develop textures of indeterminate duration—brings to the fore not just what things sound like, but how they work, and I fundamentally think there’s a real link between the way this music works and the way eating or drinking works.

Wine, perhaps more than any other comestible, brings the question of time to the fore. One of the commonly discussed attributes of a wine at a wine tasting is its finish, how long it lingers on the palette after tasting, which I’ve heard some sommeliers describe in very precise measurements. But thinking about how a sensation changes in the mouth is not limited to wine. After I gave my presentation to the R&D team at Heston Blumenthal’s Fat Duck restaurant in Bray, England, last June, I sat around with some of the chefs and played some of my musical textures, while we tasted caramel and graphed flavor profile changes over time on paper.

When I was studying computer music back in grad school at the University of Washington in Seattle, I remember encountering Dennis Smalley’s concept of spectromorphology. It was fascinating to think of a method for categorizing the different ways a sound could transform over time. Electronic musicians synthesizing sounds are familiar with the concept of an envelope, a shape that describes how a parameter changes over time (the most common variant being an Attack-Decay-Sustain-Release envelope, allowing a synthesist to specify different volume levels for different points in time), and game audio designers are familiar with the concept of a real-time parameter curve (RTPC) that links these parameters to real-time input. I have often wondered about a taste-based equivalent. Is there something about our rich musical language for working with time that can apply to the rhythms of the kitchen? How precisely can we quantify and categorize the way that taste sensations change over time? Maybe we could call it “gastromorphology.”

This issue comes up at the end of one of NPR’s recent features on Charles Spence. First Spence is quoted:

[For example,] a dark chocolate or coffee-tasting dessert, then something like Pavarotti’s [performance of Puccini’s] “Nessun Dorma,” making much more low-pitched sounds, seem to be the perfect complement to help bring out those bitter tastes in the dark chocolate or the coffee.

Then the author of the article gets in a little zinger of a last word:

Of course, ‘Nessun Dorma’ gets a little more high-pitched near the end—so there are still challenges in finding the perfect sound for a constant flavor experience.

This is exactly the problem I’m working to solve. “Nessun Dorma” does change at the end, building to a glorious climax; like most music, it evolves. This is the form of the work, and determining the form is traditionally a big part of the composer’s job. When composing for dance or film or setting text, that evolution may be hitched to another structure, but the end result is a fixed trajectory over time. And this is also why it may be problematic to use a finished piece by Debussy or Rachmaninoff in evaluating taste to music correspondences. Not only are there so many parameters to track, but the rate of change of these parameters (what we might call morphology, or, at a higher level, simply musical form) is also changing at a rate that is likely very different from the rate of change of the tasting experience.

So I return to Messiaen as a useful reference point. I sometimes find myself in the semantically awkward situation of using the antonyms “static” and “dynamic” to refer to the same thing. Sound by its nature is motion, vibration; it is dynamic. But when I talk about static music, I mean music that isn’t going anywhere, music that is nonteleological, music that is not progressing towards a specific goal. I often talk to students of game music about the challenge of taking the pre-rendered dramatic trajectory out of a piece of music they’re writing, so that the game can put it back in. Messiaen exemplifies this kind of stasis.

This project draws on a lot of ideas: Messiaen’s approach to static music, video game music that responds to user input, algorithmic or generative processes, crossmodal sensation, multimedia pairings, the rhythmic profile of a meal. Taken together, they suggest a flexible, dynamic approach to composition, and the applications are not limited to wine, or to a meal. In fact, a real-time, responsive system such as I’ve described could be equally put to use to create a customized soundtrack for any of the unpredictable events of daily life, opening up a whole new arena for creative work.

Bridging Gastronomy and Art Requires Making Connections

The Basque Culinary Center's building, which is an impressive architectural structure , in front of which is an impressive piece of contemporary sculpture.

The Basque Culinary Center is as much a monument to contemporary art as it is an incubator of ideas for contemporary gastronomy.
(All photos by Ben Houge.)

Spain has been at the forefront of contemporary cuisine for many years, rising to international prominence with Ferran Adrià’s acclaimed and highly influential restaurant elBulli, which closed in 2011. Given my work combining music and food, when I was recruited to transfer to Berklee College of Music’s new campus in Valencia in 2013, this was no small consideration in deciding to accept the gig. (I was brought on to help develop curriculum for and serve as the full time faculty member in the new Music Production, Technology, and Innovation master’s program, and I also took charge of the video game component of our Scoring for Film, Television, and Video Games master’s program.)

As a result of spending two years in Spain, I’ve learned a lot about the key figures in the fascinating gastronomy scene, a recurring theme of which is the desire to reach out and engage with ideas from other artistic disciplines. In fact, merging taste with other sensory experiences was central to the topic—La Vanguardia (the Avant-Garde)—of this year’s Diálogos de Cocina conference. Now in its sixth iteration, this biennial event, founded in 2007, is a product of the Basque Culinary Center and Euro-Toques, and from the beginning the focus has been on interdisciplinary dialog. The conference took place over two days, March 9-10, 2015, at the Basque Culinary Center’s gorgeous new building on the outskirts of San Sebastián (the city with, not coincidentally, the highest number of Michelin-starred restaurants per capita in the world). This was the first conference I had attended that was devoted solely to gastronomy—I typically find myself at video game or digital art conferences—but after experiencing the amazing dishes served up at every coffee break, this is the only kind of conference I want to attend from now on.

This is what gets served at a "coffee break" during Diálogos de Cocina

This is what gets served at a “coffee break” during Diálogos de Cocina.

The entire event was focused on the future, investigating ideas from other art forms as well as innovations in technology. Leading crossmodal psychologist Charles Spence was one of the presenters (in fact, it was he who commended the conference to my attention, following my presentation to his Crossmodal Research Laboratory at Oxford University last February), and in his fascinating talk, I finally got to hear the sound of Heston Blumenthal’s influential multimedia dish The Sound of the Sea (which was heavier on the gulls than I had expected). Adrian Cheok of the Mixed Reality Lab in Singapore shared some really wacky progress on his efforts to digitize taste and smell; one lucky volunteer got to taste one of Cheok’s digitized flavors via a device he put in his mouth. Other presentations included a history of avant-garde art, an overview of recent technological trends, and a meditation on communication in the internet age, plus panel discussions on the dining experience of the future, socializing culinary innovation, and what experimental art can bring to cooking (and vice versa).

The consistent theme was how to draw on ideas from other creative practices to enhance what’s going on in the kitchen, an investigation many of the participating chefs were already pursuing in their own restaurants.

Members of the board of the Basque Culinary Center were the main hosts of the conference, and one chef whose inviting presence was most continuously felt was Andoni Luis Aduriz of Mugaritz.  His smiling demeanor extended throughout the sessions and into a group dinner at the amazing sidrería Zelaia, where he was seen cutting enormous traditional Basque steaks for conference attendees to ensure that everyone felt welcomed. Andoni has pushed Mugaritz to explore unique collaborations over the years, working with musician Felipe Ugarte on the Mugaritz BSO (Banda Sonora Original, or Original Soundtrack) in a project that involved visits to Ethiopia and Peru, developing a food ritual with the choreographers Idoia Zabaleta and Filipa Francisco, and providing the climactic meal to a production by Barcelona-based theater group La Fura dels Baus of Titus Andronicus. After I met him and told him about my food opera project, he invited me back to give a presentation to the whole R&D team at Mugaritz, who received me with warm inquisitiveness last August.

On the other side of the country, on Spain’s northeast coast, the three Roca brothers of El Celler de Can Roca have also stretched the interdisciplinary boundaries of gastronomy, most notably with an immersive multimedia “gastropera” called El Somni (The Dream). Twelve invited guests were treated to an extravagantly high tech, one-time event on May 6, 2013. The event, documented in a film by Franc Aleu (another presenter at Diálogos de Cocina), was in twelve eclectic movements, each with music by a different composer, ranging from robotic string instruments to traditional Catalan vocalizations to neo-romantic piano, while motion captured 3D graphics were projected onto the table and onto screens surrounding the diners. Sound also plays a role in one of the dishes on their regular menu: an edible reenactment of a goal scored by soccer star Lionel Messi, who plays for the Barcelona hometown team, which comes accompanied by a recorded sports announcer commentary.

Since elBulli closed in 2011, Ferran’s brother Albert Adrià has been working to keep the family business flourishing. Currently there are five Adrià-branded restaurants in Barcelona (all located within a few blocks near the Plaça d’Espanya, dubbed El Barri Adrià), including the acclaimed Tickets, with a new restaurant named Enigma to open soon. But their most recent opening occurred last summer fifty leagues south, in the Balearic island of Ibiza, an ambitious interdisciplinary collaboration with Cirque de Soleil called Heart Ibiza. Advertised as a fusion of gastronomy, music, art, and performance, this elaborate take on dinner theater proposed a tantalizing opportunity to observe how live performance might complement a meal.

But unfortunately the meal I experienced there last summer was less than the sum of its parts. Throughout the restaurant, dancers danced, actors acted, and bodies were painted, while live video feeds illuminated the walls, but servers clashed with dancers in the aisles (resulting in having a drink spilled on me, for which the waiter did not apologize), and the performances were completely out of sync with the dining experience. Halfway through a course, the lights would dim, leaving me to munch in a lurid blue glow, which, as Charles Spence will confirm, has been shown to have a deleterious effect on food. When I could see it, the food was fantastic, served in some imaginatively sculptural tableware, such as a porcelain frog you had to kiss in order to extract the first bite, but the chaotic surroundings, completely out of sync with the meal, prohibited the performance from enhancing the dining experience. Some issues may be a result of poor logistics and layout (despite reassurances to the contrary, I missed a lot of what was happening on a stage around a corner), but I suspect that the very presence of live performers in the restaurant inherently distracts from the meal on the table.

Ibiza is also home to Paco Roncero’s infamous Sublimotion. Representatives from InHedit, the Madrid-based company that provided some of Roncero’s technology, shared the stage with Adrian Cheok at Diálogos de Cocina to discuss the use of new interfaces in gastronomy. The cost of the technology is surely one reason Sublimotion proudly proclaims itself to be the most expensive restaurant in the world, although I have to say that this elitist stance is anathema to my goals as an artist. But fortunately I already had the Sublimotion experience when I visited Ultraviolet, Paul Pariet’s innovative multimedia restaurant in Shanghai that Sublimotion has been widely derided for having plagiarized.

As Ultraviolet demonstrates, Spain does not hold a monopoly on multimedia dining experiences. Ultraviolet seats ten people per night for a twenty-two course meal in a room that has been outfitted with video projectors, a sound system, and a mechanism to waft in different smells throughout the evening. Back in 2012, my experience at Ultraviolet was wonderful, including a few truly transcendent moments. The one that sticks with me the most was the most understated: a simple slice of bread in meunière sauce with a few truffle slices, experienced in a projected forest, while subtle ambient sound played in the background, a profound synergy of the senses that remains for me a benchmark of what can be achieved in this arena.

But I must point out another course that exemplifies the pitfalls in trying to bridge the worlds of gastronomy and art. Towards the end of the meal, there was a riff on traditional gazpacho (in the world of food, Spain is never far away) that involved two different elixirs separated by an edible shot glass. As the dish was served with a lighthearted “Olé” and flamenco music began to play, the video screens metamorphosed to display not an idyllic Iberian landscape or a boisterous cervecería, but Picasso’s devastating Guernica, commemorating the horrific slaughter of civilians in a Nationalist bombing raid during the Spanish Civil War. When I first saw the painting in 1998 at the Reina Sofía Museum in Madrid, I was somberly transfixed for twenty minutes. Projecting it on the walls of a restaurant as a shorthand for Spanishness was an egregious miscalculation, insensitive to the language of art, but more important, to the tragedy depicted.

I think all of these examples suggest that, as we’re moving beyond the mere novelty of pairing sound or visuals with a meal, the focus needs to shift to what is being communicated by the resulting amalgamation; it’s not enough to simply put things side by side, to project a painting or dance in the aisles next to a diner. The languages of art and food are very different, and in many cases artists and chefs seem to be talking past each other. I think the key is to look beyond the end product of a meal or a performance or a composition to examine the processes and considerations and motivations that produced it. Despite the pervasive trend for chefs to develop dishes that visually evoke famous artworks, we should be thinking not about how a plate of food looks like a painting, but about how the work of a chef is like the work of an artist. This is where fruitful interdisciplinary conversations can occur.

At Ultraviolet, as well as in El Somni, which prominently features a 3D animation of Michelangelo’s iconic David (at one point depicted in flagrante delicto, and later shown shattering to pieces), a famous artwork is reproduced to serve as a cipher for art. I understand this impetus; for much of my career in the video game industry, I’ve observed a desire for games to be taken more seriously, to aspire to the artistic credibility and cultural respectability of film, and sometimes that means invoking the conventions of cinema in cut scenes, casting, and marketing. But in fact, in gastronomy as well as in games, these ciphers are unnecessary.

I lived in Shanghai for six years, from 2004 to 2010, and during this time, Paul Pairet was my favorite chef. When I first conceived my food opera project, back in 2006, I had his innovative cooking at his first Shanghai restaurant, Jade on 36, in mind. The food was spectacular, iconoclastic, playful, but with a serious rigor, and wildly inventive: candied foie gras on a stick, ice cream disguised in a lemon rind, sardine mousse served in tin cans. It dawned on me that the experience I was enjoying was exactly the reason I went to new music concerts, to have my preconceptions shaken, to fully engage my senses to interrogate and evaluate new stimuli, not relying on conventions of naming or presentation or other culturally learned tropes. That was when I made the leap, realizing that the kind of music system I was then designing for Ubisoft (as audio director of Tom Clancy’s EndWar) could be equally applied to the unpredictable, real-time input of the dining room, highlighting and harmonizing with the music inherent in the meal, responding to its intrinsic rhythms. Ultraviolet opened much later (in May 2012, about a week after my first food opera at Harvard), but by recognizing a parallel creative process, one that, like music, unfolds over time, I already saw the tremendous potential of pairing music with food in a way that builds on the language of each.

Having completed my two-year appointment in Valencia, I returned to Berklee’s Boston campus this fall, and I now teach music programming in the Electronic Production and Design department. One of the perks of being back in the Boston area is being able to attend Harvard’s fantastic Science and Cooking Lecture Series, which has been host to a parade of luminaries from the culinary world over the past six years. A few weeks ago I got to meet up with Andoni Luis Aduriz once again, while he and one of his chefs, Ramón Perisé, were in town to present a fascinating talk on science and emotion as part of the series.

The next day, I organized a Berklee tour for the two of them, showing them some of the fun musical gadgets we have in the EPD department before visiting the new studio facilities in Berklee’s brand new building at 160 Massachusetts Avenue. As part of the tour, my boss, EPD Department Chair Michael Bierylo, demoed a Moog System 55 synthesizer. My take on the recent resurgence of interest in modular synths is that, in comparison to the vast array of sounds available as plug-ins in today’s digital audio workstations, the constraints of an analog modular rig help focus creativity. While this may have been the first time he’d seen a Moog synth, Andoni immediately recognized this concept, the notion of freedom within limitations, as being just as true of his work in the kitchen. Identifying these kinds of correspondences is what makes working at the intersection of different practices so fascinating and why I have been incorporating these ideas into my classes, to teach students about creativity and innovation by drawing parallels to other disciplines.

The last stop on our Berklee tour was the cafeteria in our new building. The previous cafeteria (a former hotel swimming pool, I’m told) had a longstanding tradition of being converted every evening into a performance venue for student ensembles, so when designing the new cafeteria, the priorities were inverted: instead of a cafeteria that also serves as concert hall, we built a concert hall that doubles as a cafeteria. Seeing Andoni and Ramón in that space, I recognized it as a perfect embodiment of the ideals of interdisciplinary collaboration. By accommodating the concerns of two different creative practices, the potential of each is expanded, and a welcoming space emerges, awaiting unforeseeable new expressions.

Andoni and Ramón at the Berklee Cafeteria

Andoni and Ramón visiting the Berklee Cafeteria.

Tasting Notes

A group of people sitting around a long table with whisky snifters in front of them in a darkly lit room.

All the folks at this table are about to taste Chivas’s Ultimate Cask Collection, but when they do so they will also be listening, on headphones, to music that was specifically created to bring out certain taste sensations in the whisky. Can what they’re listening to affect their perception of what they are tasting? Chivas thinks so. (All photos courtesy Chivas Regal.)

A few weeks ago, I wrote some music to accompany a whisky. The premiere was the unveiling of the Chivas Regal 18 Ultimate Cask Collection at an event in London, hosted by Chivas Master Blender Colin Scott and chef Jozef Youssef. The music was not there to simply provide background ambiance to a swanky event; my objective was to change the way the whisky tastes.

Jozef Youssef is the founder of Kitchen Theory, a London-based group whose experimental pop-up dinners explore the frontiers of science and gastronomy. I met him at one of their Synaesthesia dinners last April, which drew from the growing field of crossmodal psychology to examine how the senses can overlap even in those who don’t experience classic synaesthesia.

At the whisky launch event, Youssef guided participants through a series of experiments designed to provide new perspectives on the whisky. Chivas Regal 18 is a blended Scotch whisky in which each component has been aged for at least 18 years. The Ultimate Cask Collection is a reimagining of Chivas Regal 18 that has been aged in American oak, resulting in pronounced vanilla, caramel, and orange marmalade notes. Some of these experiments one might expect at a whisky tasting, observing how dilution, temperature, and aroma affect one’s impressions. Less expected might be the texture cube I first experienced at Synaesthesia, with different materials (from velvet to velcro) on each face, allowing imbibers to experience how different tactile sensations affected their experience of taste. And similarly unorthodox was the experiment that involved listening to two musical textures while sipping; one was designed to bring out the sweet notes in the whisky, while the other emphasized its bitter qualities. Composing these textures was my job.

Jozef Youssef standing and holding a carafe of whisky as participants seating at a table look on.

Jozef Youssef guides folks through a whisky tasting featuring sound.

I actually do a lot of this kind of thing. Most of my work these days involves exploring correspondences between music and food in a series of audio-gustatory events that I call “food operas.” I’ve collaborated with a couple of chefs in recent years, and I wrote a detailed article about three events I did with Boston-area chef Jason Bond that was published in NewMusicBox in 2013. Since then, I’ve also branched out into composing music for wine and, as in the aforementioned event, whisky. Over the summer I taught a class to students in Berklee College of Music’s Music Production, Technology, and Innovation master’s program that probed the real-time technology and aesthetics behind these events; students presented their final projects alongside an eight-course tasting menu at Quique Dacosta’s Michelin-starred restaurant El Poblet in Valencia, Spain. My next food opera is a collaboration with St. Paul-based new music ensemble Zeitgeist, which will premiere in the spring.

The original inspiration for my food opera project was the idea of using real-time music deployment techniques borrowed from my work designing audio for video games over the past twenty years to score the indeterminate events of the dining room. I sought to apply recent technical advancements in games as well as ideas from the discipline of sound installation (e.g., a big multichannel speaker array to deliver sound to diners) to achieve an unprecedented level of synchronization between the senses of taste and hearing. It’s been a fascinating challenge to think about what kinds of sounds go with what kinds of foods, in a way that heightens diners’ sensory awareness of the music as well as the meal.

Since I started working on the project, I’ve become aware of a growing body of work in the field of psychology that explores crossmodal links between the senses. At the forefront of this field is a psychologist named Charles Spence, who heads up the Crossmodal Research Laboratory at Oxford University. His recent book The Perfect Meal: The Multisensory Science of Food and Cooking, co-authored with Betina Piqueras-Fiszman, is a wonderful compendium of the occasionally outlandish research being undertaken in this burgeoning field. His work has been prominently featured in the press (in The New Yorker’s current “Food Issue,” for example, and several times on NPR), and he has served as consultant to celebrity chefs including Heston Blumenthal, whose Sound of the Sea dish, introduced in 2007 at his renowned three Michelin star restaurant The Fat Duck in Bray, England, is probably the most famous pairing of sound with food. This seafood dish, plated to resemble foam on a slice of sandy beach, is served with an iPod in a conch shell, its protruding headphones immersing diners in ocean sounds as they eat, a byproduct of some of the sound pairing experiments the two conducted together. Spence has also collaborated with Jozef Youssef, and his research formed the backbone of Kitchen Theory’s Synaesthesia dinners.

Spence is widely cited for asserting that, by playing different sounds, “we’re able to show that we can change the experience in [the] mouth by about 5 or 10 percent,” as he told NPR. He goes so far as to assert that sound can be employed to fight obesity, removing sugar from foods and sonically making up for the lost sweetness.

I had a chance to meet Spence when I visited Oxford last February to present my work to his Crossmodal Research Lab. My friend Janice Wang, who helped out with my Bondir food operas, recently moved from Boston to Oxford to pursue her PhD in Spence’s group. Her own research was recently featured in the Financial Times, in an article entitled, “I Use Music to Change How Food Tastes.” Janice is also president of the Oxford University Blind Wine Tasting Society, which, at the time of my last visit, had recently administered a crushing defeat to rival Cambridge University. While I was in town, Janice and I collaborated on an event at the delightfully cozy 1855 Wine Bar in Oxford, working with sommelier Alistair Cooper to pair six different wines with six different musical textures. She also accompanied me on my visit to Kitchen Theory and introduced me to Jozef Youssef in person.

Janice is part of a group who identify themselves as Crossmodalists, committed to promoting experiences that engage and reinforce the links between all the senses. Perhaps due to the influence of figures like Spence and Blumenthal, there seems to be quite a scene for crossmodal dining in the UK at the moment, and music figures prominently. I attended one of this group’s rehearsals last April, which included a parade of scents, live piano, an edible painting, and choreography that responded to flavor—a fascinating mix.

Dining is a profoundly multi-sensory experience. A lot of times we might describe a good meal strictly in terms of taste, but in fact eating involves all of the senses, and psychologists are quantifying to what extent this is true. Smell is the most obvious supporting sense, but also the feel of the silverware, the color of the plate, the food’s appearance, even the mood of the diner, all of these elements combine to affect our perception of a meal. Even after food enters the mouth, temperature and texture merge with taste in our evaluation of a dish. In Spence and Piqueras-Fiszman’s book, I learned that we actually have two different senses of smell: the orthonasal is what we typically think of as “smell,” applying to external odors in our environment, whereas the retronasal is concerned with what is already in the mouth and is thought to have evolved independently and much earlier. This is one reason (alongside aeration) that sommeliers sometimes slurp wine, to aid in retronasal evaluation.

Sound plays a huge role in eating. Studies show that loud sounds dull our sense of taste, which may explain why airplane food tastes so bland, and why people order more tomato juice on planes, as umami flavors are the most resilient to volume. The sound of mastication also has been shown to affect our perception of an item’s freshness, improving the impression of carrots and potato chips. To me, all of these observations underscore the notion that developing sound to pair with food represents an exciting new arena for aesthetic expression.

Three people at a table wearing headphones and sipping whisky

Applying the role that sound has in affecting taste perception to drinking whisky.

My mandate for the Chivas event was very specific, and so these whisky textures represent my first attempt to write music that explicitly drew on the crossmodal psychology research. As Charles Spence will tell you, high frequency sounds have been shown to make things taste sweeter, while low frequency sounds (low brass, in Spence’s experiments) make them more bitter. My textures are short, simple, and very consistent throughout their short durations, attempting to hew closely to the script, although I still wanted to provide a bit more musical interest than a static sustained tone. They are similar enough that useful comparisons can be made where they diverge.

My sweet texture is built from recordings of a flute, a clarinet, and wind chimes. The two-voice main melodic part is in short diatonic phrases, slow and legato, a type of phrasing I often use to create a sense of peaceful, suspended time. The motion is largely stepwise, the harmonies consonant, predominantly thirds and sixths. There may be a slight evocation of the type of traditional melodies one might expect to hear in the Scottish highlands, albeit in fragments. In the background is a slow moving harmonic pad to provide a bit of context, and wind chimes tinkle throughout. There’s no steady pulse. The overall impression should be of stability and resolution with mellow timbres in a floating, high register.

In my bitter texture, the overall frequency range is much lower, as the cello takes the dominant role, playing much more aggressively and roughly than anything in the sweet texture. The cello part is broken into short, intermittent phrases, similar to the lead part in the sweet texture, and similarly in two voices, as it’s all double stops, but here there is a steady pulse, and phrases tend to emphasize more dissonant intervals. Wood blocks and temple bells further emphasize the pulse, while a low drone underneath provides a harmonic reference point. Rather than emphasize brass, as in Spence’s study, I chose to use sounds relating to wood, as the bitter elements were also linked to the whisky’s having been aged in American oak casks, and I wanted to bring out that quality as well.

There’s a question that could be raised about linking the sound of wood (woody resonance of string instruments, sound of struck wood) to the taste of wood (oak). On one hand, we might say that the taste and sound are not related at the level of sensation, that one does not necessarily evoke the other. On the other, we might observe that most humans have learned to associate these things through a lifetime of interaction with wood. The question of what’s learned or culturally conditioned as opposed to what’s innate comes up all the time in designing these pairings.

As I’ve learned more about the psychological approach to the dining experience, I’ve wondered a lot about the point at which the work of the researcher ends and the work of the artist begins. Whereas in a lab experiment, one might want to isolate certain parameters of a taste experience, a lot of times as a composer, I’m trying to blend and merge and complement. Rather than dealing with pure tastes like sweet or bitter, I’m typically interested in a complete dish that has balanced flavors and textures, thinking how music can join in as another set of ingredients. It’s usually not my objective to simply replicate a dish in sound, but to complement and transform it, guided by my experience writing music to accompany dance or video games. In a way, maybe the rules of psychological association are like the rules of music theory, serving as a reference point for a composer, a framework or a palette that can then be applied, twisted, or inverted to far ranging aesthetic ends.

A group of beakers filled with various types of Chivas whisky.

Ultimately a satisfying aesthetic experience—whether eating a meal, savoring a glass of whisky, listening to a piece of music, or participating in a synaesthetic immersion that involves all of the above—must be more than an experiment.