Mozart & The Mind
Interactive Music & the Brain Exposition
May 11, 2013
“Mainly Mozart is playing to the belief that its audience is smart and wants to be treated intelligently.” —James Chute, U~T San Diego
The Gamelan Project Alexander Khalil1, Victor Minces1, The Giri Nata Gamelan Ensemble
Interactive Music & the Brain Exposition Mozart & the Mind May 11th, 2013
You’ve arrived at an intellectual playground providing you a unique opportunity to engage with scientists, musicians, and fellow music aficionados around a series of interactive installations exploring connections between music and the brain. Here you can gain firsthand insight from scientists and artists into their fascinating interdisciplinary work, demonstrated through interactive exhibits and installations. Along the way, you will have the opportunity to experience some of the cutting-edge technology being used both to study brain function and as new medium for artistic and musical expression. As you explore these installations, we invite you to follow your curiosity – experiment, ask questions, explore your creativity while reasoning scientifically; but, most importantly, enjoy yourself!
The gamelan project investigates the relationship between ability to synchronize rhythmically and the ability to attend. An important part of this work has been to develop a reliable method for measuring synchrony amongst ensemble players. Using sensors wired to each key of each percussion instrument, each player’s rhythmic patterns are collected in isolation. These rhythmic patterns are analyzed in real time using vector strength analysis, a method usually used for measuring synchrony of firing neurons. We invite audience members to try the instruments and, with realtime feedback from the data collection system, experience what it is like to achieve a high level of synchrony.
Temporal Dynamics of Learning Center – UC San Diego
For more information: http://tdlc.ucsd.edu/gamelan Acknowledgements: This work was supported in part by NSF grant SMA 1041755 to the Temporal Dynamics of Learning Center, an NSF Science of Learning Center and San Diego Foundation; Blasker Science & Technology Grants Programtion.
Disklavier and Brain Richard Warp – Composer A semi-composed piano piece that demonstrates some of the possibilities of brain-based musical expression, and explores how music might be created that responds compellingly to the moods and intentions of the performer. Part score, part improvisation, the music moves between fixed structure and generative processes, creating a piece that evolves, but at the same time sits within certain compositional boundaries in order to achieve a sense of artistic unity. Using Max/MSP, OpenSoundControl and the Emotiv EPOC headset, brain signals (cognitive and emotional) are filtered and mapped to melodic, harmonic and timbral/dynamic parameters that are played by the Disklavier. While the ‘solo’ part of the score is composed in traditional fashion and played by the performer, the ‘duet’ portions of the score employ stochastic processes to drive generation of musical material from the instrument, while the performer improvises around this output. Disklavier and Brain is a work in progress, with a world premiere planned for fall 2013 in the San Francisco Bay Area. For more information: http://www.richardwarp.com/
BrainMovie3D Tim Mullen1, Christian Kothe1, Alejandro Ojeda1, Mike Chi2, Trevor Kerth2 “The mind is the music that neural networks play.” This quote from renowned computational neurobiologist T.J. Sejnowski underscores a growing scientific consensus that studying the structure and function of vast networks of connections between brain regions is key to understanding creativity, music, language and a host of other mental processes. BrainMovie3D provides a window into these network dynamics by computationally modeling and visualizing information flow between different regions of the brain based on electrical signals recorded from the scalp using state-ofthe-art wearable high-density EEG technology (Cognionics, Inc) and advanced signal processing. Such technology is being used to better understand the neural basis of cognition and behavior in mobile, real-world conditions. This technology may also some day be used to improve cognitive function via neurofeedback training, helping one strengthen or weaken specific functional dependencies between brain regions. This installation will be repeated May 18th and June 1st Affiliations: 1 Swartz Center for Computational Neuroscience, INC, UC San Diego 2 Cognionics, Inc. (San Diego) For more information: www.antillipsi.net www.cognionics.com
EEG Ocean: A Sea Inside A Musician’s Mind
Grace Leslie1, Tim Mullen1, Yuan-Pin Lin1
Tim Mullen1 & Nima Bigdely-Shamlo1 Performed by: Adam Tinkle2 on Saxophone, Guitar, Electronics
MoodMixer is a collaborative interactive brain-computer music interface (BCMI) installation that explores feedback loops between states of mind and music in the context of social interaction. The combined cognitive state of you and a partner are translated into a multi-channel music mixture and colored visualization. The visual and musical aesthetic at a given point in time is determined by the combined mental state of both participants. Now in its third generation, MoodMixer uses a new automatic music generator to produce a composition reminiscent of John Adams’ piano piece Phrygian Gates (1977-8). The software randomly chooses notes from a set scale and repeats them to create slowly evolving loops, hallmarks of Adams’ minimalist style, of which Phrygian Gates is a prototypical example. The scale begins in A Lydian, then shifts to A Phrygian, and then cycles around the circle of fifths, to E Lydian, E Phrygian, and so on. Several aspects of the music are manipulated to match the participants’ cognitive states, primarily tempo and mode, as these are thought to be the features that most determine a piece of music’s emotional expression. One participant’s brain controls the tempo of the piece, while the other participant influences the overall texture of the piece by expanding and contracting note lengths. Participants also have the option of jumping to the next key in the cycle by blinking their eyes. No two performances of the composition sound the same, given the unique contributions of each participant, and the piece’s always evolving structure. This installation will be repeated May 18th and June 1st Affiliations: 1 Swartz Center for Computational Neuroscience, INC, UC San Diego 2 Dept. of Music - UCSD For more information: www.antillipsi.net www.cognionics.com
EEGOcean: A Sea Inside A Musician’s Mind (ASIAMM) borrows from the cross-cultural metaphor of mind-as-ocean to create a unique visualization and live acoustic sonification of brain activity as a musician improvises live in response to an artistic interpretation of his own changing brain activity. Electrical signals originating from different parts of the musician’s brain are measured using electroencephalography (EEG) and the rhythmic electrical activity (“neural oscillations”) in different frequency bands are represented as waves rippling out from respective islands situated in a blue ocean. As the musician enters into different states of mind or attends to, and refines, specific features of the evolving composition, these changes in his mental state lead to shifts in his rhythmic brain activity. This is reflected in the waves, ripples and bursts of the ocean as well as the flocking behavior of schools of fish – which are attracted to islands with dominant activity. The musician in turn interprets this glimpse into his own brain and mind, shaping the evolving composition and closing the feedback loop. This installation will be repeated May 18th and June 1st Affiliations: 1 Swartz Center for Computational Neuroscience, INC, UC San Diego 2 Dept. of Music - UCSD For more information: http://www.antillipsi.net/art-1/bioart Acknowledgements: This demonstration is made possible by wearable EEG hardware provided by Mindo and the Brain Research Center of National Chiao Tung University, Taiwan. See http://www.mindo.com.tw/en/index.php for more information.