Subscribe via RSS Feed

Neuroaesthetics: The Gathering

[ 1 ] September 5, 2014
Top scientists and philosophers working at the intersection of art and neuroscience gather in New York City for the 2014 International Association of Empirical Aesthetics summit.
Ville cranienne (Skull City). AndreÌ Masson, 1940. Drawing on paper.

Ville cranienne (Skull City). AndreÌ Masson, 1940. Drawing on paper.

It’s hard to know how much we don’t know about the brain. The presence of vast unknowns in the field means that many current debates in neuroscience hinge on differing scales of inquiry and the significance of results from current methods. Are single neurons the place to look? How much weight do fMRI results bear? Do we actually even need to generate more data from real tissue, or is it time for large-scale brain simulation? What to spend our public billions on?

Neuroaesthetics faces similar issues, and although the findings it has offered in the past decades have gone far beyond traditional aesthetics (and I for one am a supporter), it has been attacked widely for perceived overstatements of its explanatory reach into zones traditionally under the authority of the humanities. Last weekend, at the 2014 IAEA (International Association of Empirical Aesthetics) biannual congress in New York City, current IAEA president Anjan Chatterjee seemed well aware of the ongoing criticism, and opened the meeting by providing a categorization scheme that helped clear up who is doing what, why they’re doing it, and what people know they are and aren’t doing when they conduct this kind of research and engage in its requisite collaborations. According to Chatterjee, whose recent book The Aesthetic Brain (Oxford) is a terrific primer on the field, there are three types of neuroaesthetics: depictive, descriptive, and experimental.

  • Depictive neuroaesthetics encompasses the visual depiction of neuroscience, where the images are scientifically accurate but the visual presentation of them is deliberately sculpted, such as Cajal’s classic images or more recently, Brainbow.
  • Descriptive neuroaesthetics refers to work that correlates activity in basic sensory areas of the brain with corresponding features in artworks– a sort of descriptive botany of the mind, where artworks are plucked as prototypes to discuss general neural responses– a dangerously seductive process akin to a stroll through a museum with a cognitive psychologist where each work risks being reduced to a perceptual trick, brain disorder, or similarly pre-packaged insight that relies heavily on fMRI and single-cell electrode experiments. There has been much foundational insight from this kind of work over the past decades, but its over-reliance as the rest of neuroscience continues to evolve puts descriptive neuroaesthetics in ever-weakening explanatory waters.
  • Experimental neuroaesthetics describes the maturing realms of this field, where “experiments are designed with controlled manipulation of variables of interest,” according to Chatterjee. This describes much of Chatterjee’s own work, and that of others like Ed Vessel and Jesse Prinz.

On Saturday, Art Shimamura spoke about his Dynamic Filtering Theory of the prefrontal cortex (PDF), a “grand unifying theory” that underlies the way he thinks about art and the brain. “At this stage, it’s important to look at the networks,” Shimamura observed, offering what could be taken as a credo for this next generation of experimental neuroaesthetics. Though neuroscientist Leah Kelly and I were critical of Shimamura’s recent book Experiencing Art (Oxford) in a recent review we co-authored for Science, his IAEA presentation offered a more nuanced approach to the complexity of network-level dynamics, summed up by a working mantra he shared with the audience gathered at Hunter College: “It’s a whole brain issue, stupid.”

Moments later, Ed Vessel launched into a wide-ranging and thoroughly sensational presentation, and Shimamura’s whole-brain mantra seemed to course through every slide. Vessel has established himself as one of the leaders of whole-brain experimental neuroaesthetics, connecting the act of evaluating artworks to relationships between specific regions of the brain, from perceptual and evaluative centers to areas implicated in the default mode network, the subject of an entire symposium earlier this year at Columbia (previously covered here).

After admitting the thorny issue of studying “meaning” in art (do you look at external, objective features, like symmetry and color, or at the internal subjective features, like novelty and associative meaning for the viewer?) Vessel mounted an impressive case for his method of measuring how “moving” people find works of art, whilst in a brain scanner, on a scale from 1-4. Vessel’s central scientific insight (full paper is here) demonstrates his whole-brain focus: when someone reports a work as especially moving, areas involved in the default mode network, which are usually correlated with autobiographical thought and inhibited when performing an outward, sense-dependent task, seem to “come online” while viewing these moving artworks, joining the sensory/perceptual areas and suggesting a neural signature for their personal, subjective resonance. Its intuitive logic is no mistake: this might be a first glimpse, albeit coarse and low-res, at the neuroscience of meaning.

LANGUAGE IS NOT TRANSPARENT. Mel Bochner,1970. Chalk on paint on wall, 72 x 48 inches

Chalk on paint on wall, 72 x 48 inches

One of the most electrifying talks at IAEA followed from an unlikely question: “What can conceptual art teach neuroaesthetics?” Alex Kranjec, from Duquesne University, who spends most of his time studying the psychology of prepositions, started with a personal story of coming across the artwork of Mel Bochner at the National Gallery in D.C., and finding a kindred spirit in Bochner’s own mantras, that “Prepositions hold the world together” and “Language is not transparent.”

Bochner’s work led Kranjec to a wonderful insight, which he shared with the crowd gathered at IAEA: “Both conceptual artists and neuroscientists care about visualizing thought.” Kranjec believes that scientists ought to simplify, and think more like the first conceptual artists, who were thoroughly invested in picking apart the relations between objects (very much like analogy and metaphor are used in science) and succeeded by simplification and repetition. Picking apart a Joseph Beuys piece, for example, might need to wait until these foundational, object-oriented, relational steps are taken.

“Both conceptual artists and neuroscientists care about visualizing thought.” – Alex Kranjec

But the field of neuroaesthetics wants to do more, and now. Much current work focuses on the relationship between art and affect: do the aesthetic features of artworks themselves provoke emotional responses in viewers, or are emotional responses a “downstream” event, more dependent on associate relations and subjective, personal context? Or, as Jesse Prinz’s work has explored, do certain affective responses become more likely given how we’ve been primed prior to viewing? Though it’s likely the answers are muddy mixes of the above, it was the staking of absolutist positions at IAEA in relation to emotion and art that provoked some intriguing fireworks.

It started with David Freedberg, who sees empathetic engagement with art as an almost involuntary, immediate event. Delivering a keynote address at IAEA last Saturday, he flashed a slide of a renaissance painting depicting the Virgin Mary slumping at the feet of Christ’s bloodied body, and explained that his hypothesis has always been that when we see such a work, “we notice that we have that slump within ourselves,” that we feel the physical depiction on a premotor level just as much as we purely see it. In collaboration with Vittorio Gallese, Freedberg has furthered this kind of application of mirror neuron theory to aesthetics, arguing that motor cortex-driven empathetic embodiment is a central force in the viewer’s engagement with a work of art (Other of Gallese’s collaborators have similarly applied embodiment theory to cinema).

Freedberg is developing a new model of aesthetic evaluation, which he briefly teased near the end of his presentation: essentially, that an artist’s ability to engage motor involvement is key to the success of the work initially– but it’s a crucial next step of inhibiting the motor response, which Freedberg localizes in the basal ganglia, that is necessary for the separation of viewer from viewed object, thus creating the opportunity for a more removed, evaluative moment. In this model, empathy is an essential motivating step, but it is the subsequent restraint of movement that allows for a separation of self from the artwork, so that judgement can commence. It would be interesting to know how this lines up with Ed Vessel’s model, as described above (I asked both Vessel and Freedberg for comment on this, but both felt it’s too early to draw conclusions on the relationship).

The Weather Project. Olafur Elliason, 2003. Turbine Hall, Tate Modern, London.

The Weather Project. Olafur Elliason, 2003. Turbine Hall, Tate Modern, London.

With Freedberg’s description of art as empathetic motivator hanging in the air, it was Vladimir Konecni of UCSD who took the podium and railed against the idea that art provokes direct emotional responses, a stance which provoked the largest emotional response from the gathered audience, as measured by the length and vocal tones of the Q&A that followed.

Konecni bluntly stated that he doesn’t believe paintings “sufficiently engage with the viewer’s intimate associative and memory systems,” and thus that “paintings are not sufficiently powerful to induce psychobiological states in viewers.” According to Konecni, emotions are acute, and they have unambiguous causes and objects– this is to be distinguished from moods. With paintings being insufficient to directly provoke emotions, Konecni finds more reliable responses of aesthetic awe in large installation pieces, like Elliason’s artificial sun at the Tate Modern, or Richard Serra’s work. In canceling out emotion from the discussion of artwork’s direct (a)ffects, Konecni is essentially calling for a return to Greenbergian formalism in art discourse.

The fireworks ensued when the eminent aesthetician and philosopher of art Noel Carroll challenged Konecni in the Q&A. Carroll, who had delivered the opening plenary talk a day earlier, raised a counterexample of visual art he believes does provoke direct, emotional responses: humorous or politically-charged comics. The exchange quickly devolved, with Konecni using it as an opportunity to counter-criticize Carroll’s own talk, which had focused on art as an evolutionarily adaptive behavior, and Carroll repeating the same question several more times. Though the brevity of the Q&A didn’t allow for any proper dialogue to unfold, the exchange between Konecni and Carroll was refreshingly pointed, and left me wanting to hear more of a dialogue between these two heavyweight thinkers who clearly disagree on some basic aspects of experiencing art.

The conventions of scholarly meetings serve certain needs of a field, by bringing together disparate strands of research and allowing for formal presentations of new work and informal conversations during coffee breaks, which can often lead to future collaborations. But the brief fireworks between Konecni and Carroll made me yearn for more formalized debates in these situations, so that certain other needs could be served– those of verbalizing disagreements, finding middle grounds, and casting away outdated concepts. These processes do happen, slowly and more passively, in the scholarly exchanges that occur from one paper to the next, from a book to its critical review, or from a scheduled talk to the next scheduled talk– and there is surely value to the longer durations of time involved in these conventions, where one can deliberate, write, and then deliver a well-reasoned argument. But an occasional live debate frames these processes in a shorter timespan, and gives them a safe space to play out more rapidly. For it is in pronouncing one’s difference of opinion– which surely do exist, as Chatterjee noted, from the inside and from the outside of neuroaesthetics, just like they do in neuroscience in general– where the conventions of academic politesse can be temporarily sidelined and true convictions can be put to the test. Just like recent debates regarding the Human Brain Project have given proponents and critics more opportunities to sharpen their arguments in the public sphere, some time allotted for debates at scholarly conferences could promote healthy disciplinary plasticity, and help to hone its participants’ messages to a public eager to hear what neuroscience has to say about art.

Art and the Default Mode Network

[ 3 ] February 17, 2014

A recent symposium presented by Columbia and NYU explored what happens in our brains when we’re at rest, and why those same brain regions are crucial when we view art.

“It’s not about merging disciplines,” David Freedberg told a crowd gathered at NYU’s Silver Center for Arts and Science last week, “it’s about listening.” Freedberg, an eminent art historian who serves as the director if the Italian Academy at Columbia University, was speaking about the alternately tense and productive relationship between the humanities and neurosciences. This is an intersection that he knows well: for nearly a decade, Freedberg has been collaborating with brain scientists— notably Vittorio Gallese in Parma, Italy, one of the discoverers of mirror neurons— but his comments dutifully acknowledged the turf battles that such interdisciplinary strivings continue to stir up, all too often hijacking productive dialogues and turning them into negative claims and calls for retreat back to party lines.

Freedberg went on, recalling a recent meeting with neuroscientist Edward Vessel of NYU, in which the two were discussing potential avenues of collaboration. Vessel mentioned some of his own recent work, which has probed a set of associated brain regions known as the default mode network, and its relationship to aesthetic evaluation of artwork. Freedberg hadn’t heard of the DMN (as I hadn’t either), and he recalled Vessel’s blunt reply: “Then you’re behind.”

It didn’t take long for Freedberg to catch up. He wisely made the default mode network and its relation to aesthetics and creativity the subject of the Italian Academy’s annual symposium, jointly held last week by NYU and Columbia. In focusing on the DMN, an increasingly popular area of brain research, the symposium vaults Freedberg’s Academy back to the forefront of the interdisciplinary dialogue around art and neuroscience—and the forefront is a familiar position: for nearly a decade, the Academy has organized symposia and supported the advanced research of scholars from the sciences and humanities alike, uniquely focused on fostering cross-discipline “listening” at every step. Past symposia have included Art and the New Biology of Mind in 2006, which featured Antonio Damasio, V.S. Ramachandran, Semir Zeki, Margaret Livingstone and other heavyweights of the early days of neuroaesthetics, and Vision, Attention and Emotion in 2008, which did indeed receive attention in the form of a column in the New York Times.

Here’s Freedberg interviewed in 2006 about “bridging the two cultures”:


Despite its rise in popularity among brain researchers, the default mode network is surely an unorthodox choice for a symposium on neuroscience and its relation to the humanities: the scientific work on the DMN is still so young, and thus the interdisciplinary threads connecting science and art in this domain still few and far between. But it is a progressive choice, if only because the field of neuroaesthetics is in need of an injection of complexity. Looking back, the pioneers of neuroaesthetics mounted the findings from basic visual neuroscience, be it color processing in the retina to line and motion processing higher up in V1, to discuss hallmark works of abstraction and impressionism, and for good reason: these are some of the clearest findings that neuroscience can offer for discussions of basic aesthetic features of art. But as is intuitively felt by any interested in this work, the perception, evaluation, and creation of art involves so much more than basic sensory systems, and thus the new frontier of interdisciplinary work is approaching issues like embodied simulation (involving the motor system), emotional priming, and, in this case, the default mode network.

Many of the talks at the Italian Academy’s symposium were necessarily definitional: what is the DMN? What regions are involved, and what does that indicate about function? Having coined the term himself and written seminal papers in the early 2000s elucidating its function, Marcus Raichle of Washington University’s School of Medecine is considered to be the godfather of this field, and was to deliver the keynote address following Freedberg’s introduction. But after health issues grounded Raichle in St. Louis, Bill Kelley of Dartmouth (a former student of Raichle’s) filled in, and leaned on some eloquent video interview clips with Raichle to present a definition of the DMN.


In one such clip, Raichle put it this way: “There is a paradigm shift going on. The focus has been on getting the brain to do things, rather than studying what it’s doing all the time.” In approaching the DMN, Raichle’s musings demand we reorient our binary notions of active versus inactive, for with the DMN we find the omnipresent “baseline” brain, the parts that brain imaging studies always seek to cancel out so that the true point of “activation” can be seen.

So how to we approach a true definition of the DMN? It turns out a single definition is hard to reach, but broadly speaking, that we’re talking about a network of regions in the brain which are most active in the moments when we aren’t performing any outward-directed task. As Raichle explains, this “default mode” consumes most of the energy metabolized by the brain as a whole. It’s an omnipresent, baseline state, but it is most active during the in-between moments when you’re staring up at the ceiling, riding on the train, reflecting at the end of a long day– these moments seem to be tethered in study after study to activity in regions such as the precuneus, the anterior cingulate cortex, and the inferior prefrontal cortex, which are regions that have been implicated in authobiographical thinking, and in the relation of the self to other people, events, and planning for the future. But these are early days of understanding the behavior of this network, and those are large areas of the brain to be tossing around in any kind of explanatory way. Daniel Margulies (Max Planck Institute, Leipzig) and Felicity Callard (Durham U.) wisely delivered a comprehensive overview of the current definitional status of the field, and urged a more specific analysis of the subunits of the default mode network and their functional roles within the network as a whole. To ask if the anterior cingulated cortex is involved, Margulies offered as an example, “is like asking where the best coffee in the world is and being told ‘Europe’. It is much more helpful to get information at least on a country or a city, and that’s the same as with functional roles of areas implicated in the DMN.”

What seems to be clear is that when we engage in any kind of task that snaps us out of the familiar autobiographical background of our thoughts, this network dips in activity, inhibited by new regions that come online to deal with the outward behavior. As Bill Kelley told the audience at NYU’s Silver Center, it’s more and more seeming like the DMN could be the primary seat of personality, and that “differences in personality among individuals may manifest as differences in resting-state default mode network connectivity patterns.” Yvette Sheline of UPenn added detail to this principle, relating findings from her work on depressive brains that reveal greater connectivity between default mode network regions involved in autobiographical thought and other regions involved in the judgement of others, suggesting a basis for the tendency of depressive patients to relate negative valuations to the self.

In addition to differences in personality, there is growing appreciation of the DMN as a major structural and functional leap from primate brains to human brains. Randy Buckner of Harvard could barely contain his enthusiasm for this line of work, telling the audience assembled the following day at the Italian Academy on Columbia’s Morningside Campus that “we feel lucky to have stumbled on something so interesting. We have jobs for the indefinite future.”

The proportion of sensorimotor cortices stays relatively unchanged through evolution; everything else expands in the human brain.

Buckner delivered a thorough and eloquent tour of human evolution that focused on the DMN’s position in the evolution of the primate brain. It was the shift in attention from a full focus on the outside world to a split focus on external stimuli and internal world-modeling, Buckner theorizes, that accounted for the cognitive leaps that eventually separated modern man from his ape ancestors. Just look at how the physical proportions of the brain have evolved: primary sensory systems are about the same size in apes as in humans, but all of association cortex— including the prefrontal cortex, which usually gets most of the evolutionary attention— has expanded dramatically. Buckner believes the “nether regions” of association cortex are what truly distinguishes the human brain—folds upon folds of cortical sheets devoted to nothing in particular, tethered to no sensorimotor pathways, that can be sucked up during development— sometimes even decades into a lifetime— making them the hallmark of cultural learning. It’s these reverberant circuits of association cortex that are wired to themselves which allow us to be flexible, highly plastic and adaptive, and which make up the bulk of the DMN, so that, as Buckner joked, “we can think to ourselves and worry all day.”

Buckner’s evolutionary tale brings us back to the present: what is research into the DMN revealing about the self, creativity, and aesthetics? The organization of the symposium, mirrored in the organization of this blog post, left any direct implications on art and aesthetics until the final talk, when Ed Vessel took the podium and worried that “the day has given us tools to tear his aesthetics work apart.”

Vessel considers his work to be a step forward from the aforementioned “first wave” of correlational studies in the field of neuroaesthetics, which have looked at single areas of the brain reacting to familiar works of art (this kind of work is what Anjan Chatterjee called “descriptive neuroaesthetics” in his recent book, The Aesthetic Brain). To move away from single areas and understand aesthetic response at a network level, Vessel has naturally latched onto the rising wave of DMN studies, curious about how aesthetic response might interact with the autobiographical, background brain.

To do this, Vessel devised an appraisal system for viewers to rate a wide range of artworks—from abstraction to portraiture and landscapes—while laying in an fMRI scanner (here is the paper, published in Frontiers in Neuroscience). The participants were shown the artwork for a brief interval, then given four seconds to submit a rating on a scale of 1-4 of how powerful, pleasing, and profound they found the image. Vessel’s key finding is that for ratings of 1-3, the DMN showed fairly low activity, with subtle, linear increases as evaluations improved. But for ratings of 4, there was a dramatic, step-like jump in activity, as if the DMN fully “came online” for the highest aesthetic appraisals. At these moments, the sensory areas involved in viewing the art stayed online as well—a rare co-activation of two networks that usually exhibit toggling behavior.

The DMN comes online with a rating of “4”, seen in the bottom right figure. (From Art reaches within: aesthetic experience, the self and the default mode network, Vessel et al., Frontiers in Neurosci, Dec. 2013).

Vessel described this step-like activation of the DMN during “4” ratings as a “signature” of aesthetic response, and argued that this activity supports the notion that the DMN is about self-referential processing, as in, “I love this painting.” Though further experiments may benefit from more specific rating scales (Vessel told me he’s now working with a sliding scale, where participants mark their response using the position of a horizontal slider rather than a fixed number) and perhaps be more conscious of the confounding effects of any kind of appraisal rendered in the scanner (perhaps participants should just be scanned while looking at artworks, then give their appraisals later when they’re out of the scanner, so that a reverse correlation can line up their appraisals with the brain activity they displayed upon seeing each work), the results are intriguing and do point towards the DMN’s central role in aesthetic response.

It’s important to look at the conclusions from such a study, and to ask questions from the point of view of the humanities: what has this study told us about the way we view and evaluate art? Is there anything new to be learned about what happens in the brain at these moments? Though the peak aesthetic response did correlate with that tell-tale jump in DMN activity across subjects, Vessel’s data reveals that there was very low agreement across subjects in their emotional responses to the paintings he presented them: people did not agree on their favorites. So one of the study’s findings that Vessel announced to the audience was that “taste in art is highly individual, yet highly felt.” Another conclusion he offered was that “art can feel strikingly personal.” And, when it comes to the spike in DMN activity, that “self-relevance is an integral part of aesthetic experience.” For someone coming at this from the humanities, the big takeaways still end up sounding like self-evident, intuitive truths known to the arts for eons. Vessel’s conclusions, stated in this manner, sound like the findings of an alien society trying to come to terms with this thing humans do called “art.” It would be prudent for researchers presenting their findings on aesthetics to have a lighter touch with their conclusions, if only to avoid the turf battles that so plague the current landscape. For researchers like Vessel, the results may speak volumes for the neuroscience of the DMN and its relationship to aesthetic appraisal; for art theory, these statements can sound like counting to ten.

The bottom line is that though the neuroscience of network-level activity like the DMN may still be far too young for any meaningful, novel contributions to aesthetics, the research must march on if we do eventually hope to break new ground. As David Freedberg offered in his conclusion to the symposium, “The humanities don’t really know about what happens in the brain—we can just look at the results from neuroscience about aesthetics.” So to avoid turf battles, the directive to the humanities might remain as Freedberg stated it: “look and listen,” but to the scientist speaking to the humanist, we might say, “do tell, but tell us what we don’t already know.” For an emerging field of research into the default mode network, which appears to be a seat of autobiographical thought, the stakes for interdisciplinary dialogues may be higher than ever, and thus an awareness of domains– what they know and do not know, what they can offer and what they crave– is more important than ever.




Interview: Michele Guerra

[ 1 ] July 12, 2013

Michele Guerra

Michele Guerra is Assistant Professor of Cinema at the University of Parma (Italy), where he conducts research alongside Vittorio Gallese. He is the author of “Il meccanismo indifferente. La concezione della Storia nel cinema di Stanley Kubrick” (Roma, 2007) and “Gli ultimi fuochi. Cinema italiano e mondo contadino dal fascismo agli anni Settanta” (Roma, 2010), and the co-editor of “Sequenze. Quaderni di cinema 1949-1951″ (2009) and “Le immagini tradotte. Usi Passaggi Trasformazioni” (2011, with a Preface by Linda Hutcheon). Now Guerra is at work on a project on cognitive neuroscience and film experience, using the concept of “embodied simulation,” (ES) whereby the observed and felt state of another is embodied–or mirrored– by one’s own cognitive system. This developing research marks a novel, cross-disciplinary approach to film analysis and film theory. Guerra also participated in the Mellon Summer Workshop 2011 at the Center for the Advanced Study in the Behavioral Sciences (CASBS) at Stanford University on “Cognitive Science and Neuroscience for the Humanities”.

NH: Your work draws parallels not only between the body of the viewer and that of the viewed subject, but also the body of the filmmaker, so I imagine that everyone in the cinematic transaction might be interested. Why do you think casual viewers of films are interested in your work, and for what reasons are filmmakers, or actors, interested? Are there aspects of the work that speak to the creative process as well as the viewing process?

MG: We know that the power of cinema lays in its ability to reproduce something very similar to the real. When we are at the movies we face people who look like those we run into in our everyday life, we see places that look like those we inhabit or pass through daily, and we have the impression to be part of what we see, to be present to a fictitious world. Nonetheless, this basic form of realism – that is mere inferring something about it, cannot satisfy the viewer: film experience is a multimodal experience. To grasp characters’ actions, intentions, and feelings, the viewer has to share something more. He/she has to move inside the story with them, has to share such actions, intentions, and feelings, eventually to “resonate” with them, in order to be fully part of the story and to comprehend it.

Film style basically pursues and provides this impression, by promoting a motor approach to the story and by reinforcing the viewer’s virtual agency. The acting brain, as the discoverers of mirror neurons taught us, is above all a brain that understands. Though we are seat and still in the dark, we move inside the story: the camera is able to make us walk, run, jump, turn, sometimes even fly. Without this low level of resonance, it would be impossible to get the higher cognitive levels of film comprehension. To borrow Vittorio Gallese’s expression, I would say that we should implement the classical Theory of Mind with another one more focused on the Feeling of the Body. Roughly speaking, this is also, nowadays, the relation between cognitive film studies and neuro-cognitive film studies.

An approach to film studies based on ES (embodied simulation) aims to focus on the motor components of film experience. The motor programs elicited by the movies are rather neglected within film theory, and when they are taken into consideration, they are often not supported by neuroscientific data. However, there is a tradition of film physiology that, at the beginning of the 21st century, focused on these aspects. Think of Edouard Toulouse, a French physician who, in 1920, wrote that the perception of the movement on the screen triggers the correspondent movement in the viewer, foreseeing, to some extent, the action of mirror neurons. We find very similar statements in Eisenstein, related both to film acting and film style. In short, film style and techniques play the main role in making the viewer able to interact with the movie.

Finally, who cares about a research like that? And why? Talking with casual moviegoers I have always found great interest in reflecting on their responses to moving images, and huge curiosity in discovering that their film experience would rely on pre-cognitive mechanisms rooted in their brain-body system. Filmmakers and actors are much more interested in this research than the majority of film scholars, who are often skeptical about this branch of studies. Some years ago James Cameron used fMRI to evaluate the impact of Avatar’s trailers on the viewers, while Steven Soderbergh and his screenwriter Scott Burns admitted in an interview their interest in making a movie on mirror neurons. Gilles Deleuze’s prophecy comes to mind: “I don’t believe that linguistics and psychoanalysis offer a great deal to the cinema. On the contrary, the biology of the brain does.”

NH: If the mind is a body moving through space, are you investigating aspects of filmmaking having to do with different types of camera motion, editing rhythms, field of view, not to mention music? In other words, will the research program of ES take us to all corners of cinema, beyond the study of emotional response to persons onscreen?

MG: Our main goal is to demonstrate how cinema can contact the viewer at the embodied level. On the one hand, the movie can exert a very strong control on the viewer’s gaze behavior, kinesthetic sensations, emotions; on the other it can leave the viewer free of wandering across the shots, by weakening the embodied potentiality of film style and enhancing subjective feelings – some would describe it as the main difference between commercial and non-commercial cinema…

Motor responses play a key-role in the very first phase of our film comprehension, and film style and editing basically aim to guide and orient these responses, helping us in building and reinforcing our cinematic spatial cognition. The field is really large, and neuroscientific studies of film are in their first infancy. Nonetheless, I think this is the only way to say something new on the nature of the moving images, which nowadays are very relevant even – or mainly – outside the movie theater. Obviously, what film scholars would need are neuroscientists curious about these topics, and perhaps convinced that the study of our aesthetic experience might contribute to their own research. A few years ago, the American phenomenologist Don Ihde wrote a short book entitled Embodied Technics, in which he argues that our contemporary technologies embody–or in some cases re-embody– our fleshly experience in new ways, in interactive ways. I consider the ES theory a good stepping-stone to start a study of the embodied relationship with technologies and then with styles that are obviously related to those technologies.

NH: I read in an interview that you once studied and wrote about Stanley Kubrick. I imagine one could spend a lifetime studying Kubrick with ES experiments; from your perspective, since you’ve been doing ES work, do you have any hunches as to what some of the distinct qualities are that make his films so unique in tone?

MG: My interest in Stanley Kubrick’s cinema is mainly due to his extraordinary attention to film technologies. He started as a photographer and in many interviews he said that he was very interesting in understanding the “nature” of his camera – i.e. its functioning – in order to take good pictures. Kubrick never separated the story from the technique through which he had to narrate such a story. Every movie is, to some extent, a test on the potentiality of film style and technology: think of Paths of Glory, with its Max Ophüls-like camera movements in the chateau scenes, the documentary-like scenes in the trenches, with the camera that follows the characters in those absurd hallways (and the hallway will become a classical figure in Kubrick’s style), and the geometrical scene during the soldiers’ trial. Or think of the audiovisual symphony of 2001, or the use of hand-camera and wide-angle lens in A Clockwork Orange, the zoom and the lighting in Barry Lyndon (with the super-fast 50mm lenses originally produced by Zeiss for NASA), the steadicam in The Shining, or the tracking shots and the handheld camera that characterize respectively the two different parts of Full Metal Jacket.

I am pretty sure that there is no filmmaker more aware than Kubrick that to narrate the story of our being-in-the-world, you have firstly to find a way to give shape to our being-in-the-cinema. Kubrick’s movies are loved both by film critics and the audience because he is able to make us perfectly interact with them; the audience simply enjoys such interaction, the critics enjoy it too and appreciate how it has been reached. We have not worked on Kubrick yet, but when I had to choose a picture for a post on the motor approach to film style published on the website of the Society for Cognitive Studies of the Moving Image, I chose without hesitation little Danny Torrance on his tricycle in the hallways of the Overlook Hotel.

NH: As I’m sure you’re aware, there are those who come across this work, or who may read this, and raise protest to the supposed goal of this research, to reveal why we love films, what empathy is in the brain– the ultimate, deep questions. And sometimes these skeptics then raise questions about the worth of it all– whether the work will get us anywhere. Do you come across such skeptics? What is your response, and what really is the ambition of what you’re hoping this work will reveal?

MG: Of course I have come across neuro-skeptics many times. To me neuro-skepticism has to be considered as a precious antidote against what I would describe like “magic neuroscience”. Neuro-skepticism is the natural reaction to the dangers of reductionism and to the idea that cognitive neuroscience would oversimplify the complexity of experience. There are books written by neuropsychologists in which we notice skepticism about the possibility for neuroscience to have a real impact on economics, law, ethics and morality, or even aesthetics (see Carlo Umiltà and Paolo Legrenzi’s Neuromania).

Though we have many excellent works by several distinguished neuroscientists who have demonstrated how neuroscientific insights can corroborate and in some cases revolutionize positions in those fields of research, I obviously respect a skepticism based on a deep knowledge of the brain-body system and related literature. Cognitive neuroscience is a young branch of research and we cannot expect a wide sharing when it crosses fields of study that have a longer and more complex tradition. In the humanities every approach – semiotic, psychoanalytic, structuralist, postmodern, etc. – is subjected to skepticism. What is harder to be accepted is a skepticism just based on prejudice, or on some newspapers’ headlines that trivialize serious research. The fear of reductionism, as well as the fear of a confrontation with science are not only unfounded, but also signs of cultural restriction.

Obviously we need to develop a critical neuroscience, i.e. – according to Shaun Gallagher – a cognitive neuroscience that is non-reifying and non-reductionist. Neuroscientists I work with, or I talk to are not “neuromaniacs”, they are perfectly aware of the limits of their research and they are much more prudent of their colleagues within the humanities in applying their methods to other disciplines. We live in a cultural phase in which the crisis of the so-called “Grand Theory” within the humanities and the rising of a new form of humanism within other domains like biology or physics is challenging our thought. If one is really interested in the human being – like the term “humanities” suggests – or if he/she is really interested in what makes us human – and the arts are of course a crucial component – we cannot pretend that research focused on our being-in-the-world are, to say, out of our reach.

You mentioned empathy: is it possible to study empathy – a relevant matter in film studies – without referring to cognitive neuroscience, and more specifically to ES? Obviously we have to keep our identity, we do not have to become neuroscientists. Our goal is to integrate our theories and our research with data suitable to improve our knowledge. We do not have to say something about memory, physiology of vision, motor behavior, gaze behavior per se, but we have to wonder how cinema plays with these aspects of our understanding and how eventually it shapes them. And yet, neuroscience implies a totally different approach. We are not used to experiments that take a lot of time, and to progress step by step answering apparently small questions in comparison to the great queries of our traditional way of speculating. Nonetheless, it is blatantly clear the relevance of cognitive neuroscience for psychology, cognitive science, phenomenology, and even aesthetics.

My reply to skeptics could be this: try to interrogate the movie without recurring to theoretical paradigms elaborated elsewhere. Try to understand at which level it contacts you. Try to deal with it concretely. You will get back to your mind, and then to your brain-body system. Sure, in such a survey you will also find your biography, your education, your own memories, the cultural environment that shapes your thought, and you will appropriate the movie through all these aspects, and nonetheless the movie and its author were not originally interested in such appropriation, they did not work on your cultural specificity, they primarily work on a primordial contact shaped by perception and non-propositional formats of comprehension. Aesthetic experiences are to be considered like complete forms of gradual construction of the self. I agree with Mark Johnson that we need to concentrate on the bodily depths of human meaning-making, always groping toward the sense of our identity, a sense that cannot be conceived of outside the multilayered dimensions of embodiment.

Interview: Idan Segev

[ 3 ] July 11, 2013

Idan Segev

Idan Segev is Professor of Computational Neuroscience and former director of the Interdisciplinary Center for Neural Computation (ICNC) at the Hebrew University of Jerusalem, where he received his B.Sc. (1973) in Math and Ph.D. (1982) in Experimental and Theoretical Neurobiology. His research team utilizes computational and theoretical tools to study how neurons, the elementary microchips of the brain, compute and dynamically adapt to our ever-changing environment.

In recent years, Segev‘s laboratory has worked jointly with several experimental groups worldwide in an endeavor to model in detail the cortical column. Segev also takes a keen interest in the connection between art and the brain: he served as editor of a 2011 special edition of the journal Frontiers entitled “Brain and Art.” He has also recently edited an art book with original etchings by ten Israeli artists, prompted by an encounter with ICNC researchers.

NH: I am interested in your perspective on collaborations between artists and scientists. Sometimes there can be friction between the two groups, if either feel that the other is overstepping or talking about something unimportant. I am sure you’ve seen the other side too, where collaborations lead to something new and welcome, or even just new questions. Do you think it’s a good idea for artists and scientists to collaborate to some degree, and what’s your current thinking on how that’s best done and what it can result in, based on what you’ve participated in and witnessed?

IS: I agree that there is inherent tension among these “two cultures,” mostly because they do not understand each other. Their language and styles are different. But these are the two of the most creative groups,and I believe in 50/50 sharing in meetings among them, with discussion ranging from introspection, the process of having a new idea, on the role of memory in creativity, on the role of movement (in creating a piece of art) and, more generally, what is the purpose of each field, when do you say that “this is good” – are all essential subjects to discuss. Such discussion – if successful, will lead to new directions in the brain of the involved groups.

NH: What did you hope to do when you began working on the special Brain and Art Frontiers edition that you edited? And now, generally speaking, when you look back on the wide array of approaches, levels of analysis, and insights throughout all the papers, what did the process of editing the collection teach you about where we are today in exploring the relationship between art and the brain?

IS: It was indeed a “long shot” with the aim, again, on trying to bring dialogue to these “two cultures.” I am hopeful that when this special topic in Frontiers will come out as a unified ebook, in a few months, and when enough people will look at it–thenm in the brain of the reader, this harmful separation between the arts and the sciences will be reduced–that the reader could find similarities in the search for “understanding,” and that he or she will be inspired by commonalities–that in the future art and science museums will collaborate in the same physical building.

NH: Related to the issue of collaboration, there are still those both on the scientific side and from the humanities who deem these interdisciplinary pursuits of brain and art to be misguided, overly reductionistic, or overly romanticized. Just as “hip” as it is to study art in the context of the brain, more and more it seems hip to criticize these pursuits and proclaim that science will never answer or solve the deep questions of human subjectivity. Do you encounter these sorts of reactions and proclamations? What is your attitude on the matter?

IS: I do, and I agree that brain sciences are limited in their pretensions. We (brain scientists as such) are not in the position to, for example, give meaning to things (moral, ethical, etc.)– but we can highlight the physical/biological underpinnings of these (the genetic/brain basis for moral decisions, etc.). So we are not aiming at “taking over” the humanities and the social sciences; rather, we want to understand the physical basis for these (and, after all, the brain, the genes, are all physical elements– no “miracles” or “spirits” run in our heads, but rather electrical spikes and chemicals and molecules). So we will eventually get to the basis of all human actions from the physical perspective (namely Science)– then people will continue (of course) to write poetry, believe in God, or in free will (even if they will know that we are just physics).

NH: I noticed on your website that you call neurons the “aesthetic elementary microchips of the brain.” Why is the aesthetic quality of neurons significant to you, and what roll does your enthusiasm for the dialogue between art and neuroscience play in your daily research at your lab?

IS: Indeed– as one looks at trees in a beautiful landscape, one looks at neurons and sees how intricate their varieties of shapes are. Especially now, with “Brainbow”-like genetic techniques, where one can see these cells (in vivo) stained with a palette of colors, one may use neurons to appreciate their “artistic” quality, for example by having exhibitions based on these techniques. See “the Color of Thoughts” exhibit that is now running in Europe via our Center in Jerusalem.

NH: You’ve just completed a series of lectures for Coursera. With worldwide distribution of high-level academic resources making scientific knowledge more and more readily accessible, how do you hope these new tools will improve education?

IS: This Coursera and alike e-lectures (or MOOC, as they call it) is an amazing new jump in disseminating astute knowledge. It will completely change how Universities function. Getting responses from isolated places, by people who would never be able to experience a serious set of lectures, is amazing. I am very pleased by this direction, and happy to get these compliments and enthusiastic responses. I feel lucky to be in this age of turning points, whereby you can sit alone and learn in the deepest sense. The only thing that is missing, of course, is direct eye-to-eye interaction.

Interview: Julia Buntaine

[ 35 ] July 10, 2013

Julia Buntaine

Julia Buntaine‘s artwork provides conceptual footholds for issues in neuroscience: from brain imaging, to structural organization and functional pathways, her exploration of the mind is simultaneously playful and deeply intelligent. Born in Massachusetts, Buntaine attained her BA from Hampshire College, her post-baccalaureate certificate from Maryland Institute College of Art, and studied at the School of Visual Arts in New York. Buntaine has been featured in a variety of exhibitions at Hampshire College, MICA and SVA, has a permanent outdoor work on the grounds of Hampshire, a work in the Johns Hopkins University’s private collection, and two works in Subjective Resonance Imaging, The Beautiful Brain’s recent group gallery show organized in collaboration with the Neuro Bureau as part of the 2013 Human Brain Mapping Conference in Seattle. The artist is currently working and living in New York City.

NH: I’m interested in when you became interested in neuroscience. When did that happen, and what did it?

JB: For as long as I can remember I have been interested in being an artist. I attended an arts high school, and when I wasn’t making art I was exploring abandoned mental hospitals with a group of friends. In wandering the derelict campuses of these outmoded institutions, I developed an interest in the history and mechanisms of mental illness. When I began at Hampshire College, I thought to take the biological approach to studying the mind, and realized very quickly I had found something that would rival my love for creating art. My interest in the psychology of mental illness was replaced with my fascination of the biology of the brain – thinking about the self as being a byproduct of actual physiological processes agreed with my sensibilities more than any other perspective had before. I took comfort in the solidity of the scientific facts. Neuroscience was not only answering my questions but enabling me to ask more, and by the end of college I was deep into the cellular and molecular studies of the brain. It was also by the end of college that I combined my love for art and neuroscience by making neuroscience my artistic subject matter, which in turn gave my art practice a purpose I had felt was lacking previously.


“Empire State of Mind” by Julia Buntaine.

“Brodmann’s Subways”


NH: You have made pieces inspired by fMRI images, circuit diagrams, Brodmann’s maps, and specific anatomical structures. As a bio-artist, what do you hope that these pieces can communicate about the underlying theories, imaging techniques, or physical structures? More generally, how do you view the role of the artist in the relationship between the public and the scientist?

JB: I believe the discoveries of biology, among other sciences, can and should be shared through the medium of art. Art inherently demands subjective judgment and interpretation, and unlike reading articles or looking at data charts, concepts embodied in art are understood without words. Many scientific topics often have the reputation of being too complex to understand, and in my work I hope to act as a sort of interpreter, to provide my audience an alternative way to understand the wonders of biology we have discovered in ourselves. There are many techniques to approach the relay of scientific information in a visual, creative manner, and I often end up mixing scientific imagery with more familiar forms to make the subject more reachable for the audience. In each work I always begin with a strong grounding in science and then depart into the world of aesthetics as I manipulate the idea through the use of scale, metaphor, material and form.

NH: I am curious about how your work and your mind have co-evolved since you became interested in neuroscience. Looking back on the pieces you’ve made so far, do they now serve as visual reference points for you, when you imagine the world between our ears, and the techniques we use to image it? Has the process of exploring these issues through your art evolved your understanding of the brain?

JB: Making my work has always aided in my understanding of the science behind it; in fact, it was through my academic pursuit of neuroscience that I had the idea to create art about biological topics in the first place. During my junior year at Hampshire I was taking a broader themed biology course and found myself making lots of flashcards about human physiology. I remember I was going over the way blood is pumped through the heart, drawing arrows through this valve and that, and then I picked up some pencils and started to color and shade the edges… and then it had been twenty minutes and I had not only a nice colored pencil drawing, but a complete understanding, an actual feel for the movement of blood through the heart. Soon after I began to experiment with brain based sculptures, and knew I had found not only my best method of studying, but a subject matter I felt that mattered and truly cared about. Still now, in the way I could “feel” the path of blood through the heart because I had drawn it, I feel that I understand, in a bodily way, each biological subject of my works. This other type of understanding, beyond the scientific and diagrammatic, is the experience I strive to give my viewers.

NH: What is your attitude towards direct collaborations between artists and scientists? What do each have the potential to learn from the other, and what results can come out of it?

JB: As a science-based artist, I am not terribly interested in the idea of collaboration with science. Conceptually, I find there is an inherent issue in the idea of a melding between the two, which has to do with the foundations and methodologies of each given discipline. Say a scientist and an artist do truly collaborate… The bedrock of scientific discovery is the rule laden methodical testing of historically embedded hypotheses, and when scientific breakthroughs occur, it is by individuals who so completely understand the subject matter as to be able to see beyond its current confinements. The bedrock of artistic creation is based solely on what is important to the artist at the time, be it art history, what colors she has on hand, what was in the morning paper, or the like. In a true collaboration, what would result could be never be considered science, but could be art (if the artist says it is). And with this conclusion we are at the one-way road running from science to art, the road that already underlies science-based art today.

That is not to say that science can’t gain anything from art. Specifically, art made about a particular scientific topic could give scientists a new way of explaining their work, much in the way that science journalists create metaphors to relay scientific findings to the public. Science-based artists create visual metaphors to do the same, and as an artist who makes such work I hope that I benefit science by such means.

NH: I understand that in addition to your artwork you are also in the process of launching a new science-based online art magazine. What is your hope with this project?

JB: In coming into my own as an artist, understanding my place in the art world and facing my unique, subject matter-based challenges, I have kept a running list of artists I know about or meet who have similar experiences to me because of their scientific subject matter. Although it is true that many artists have been interested in science every now and then, it has only been for about the past two decades or so that science has been treated as a serious, lifelong subject matter for artists. In my search for compatriots, I found that most contemporary artists I’ve been following are based in London or Berlin, which struck me as odd given that the United States is not only home to the art capitol of the world, but to many other artistically important cities including Miami, Chicago and San Francisco. I realized that here, maybe there is not a lack of science-based artists, only a lack of infrastructure to support us. In my publication SciArt in America – set to launch late summer 2013 – I am going to feature artists and post opportunities and SciArt news within the States, have a place to discuss our common issues, and create the potential for connections among science-based artists, in an attempt to begin forming a real science-based art community.

Visit the artist’s website for more.


Artists and Scientists in Dialogue at the Rubin Museum of Art

[ 0 ] February 16, 2013

The sixth annual Brainwave series kicked off on February 6 at the Rubin Museum of Art in New York City, and will run through April.

François Girard and Carl Schoonover at Brainwave. (Photo: Michael Palma for RMA)

Just before neuroscientist Carl Schoonover and director François Girard took the stage last Wednesday at the Rubin Museum of Art in New York City for a one-hour public dialogue, series curator Tim McHenry told the crowd that the two had never met before. A murmor of surprised excitement spread through the Rubin’s basement auditorium– we were about to witness that first meeting of minds, and the respective practices of art and science carried within them (opera and film in Girard’s case and neuroscience in Schoonover’s, who had just handed in his doctoral thesis at Columbia University, and last year published his first book, Portraits of the Mind).

What ensued was somewhat like two masterful musicians taking the stage to improvise a set: first there was a warm up period, as the two got to know each other, tested the waters and lobbed some preliminary questions back and forth; then the interplay of ideas began to soar, giving the assembled audience a dialogue that seemed to be more than the sum of its parts.

Girard was in town to direct Parsifal at the Met Opera. Schoonover, bring an opera enthusiast himself, was curious to explore the question of reality, in everyday existence and in created worlds, especially Wagnerian ones. This iteration of the Brainwave series bears the general theme of “illusion,” so starting with questions of reality in the imagined worlds of opera seemed appropriate.

“Wagner’s worlds were not literal, they weren’t real in the sense that you thought they happened,” Schoonover said to Girard. “Would you argue that those worlds hold as much sway in your life, in the way you make decisions in the world, as say, getting kicked in the toe? Do they have the same immediacy in your life?”

Girard replied: “All I’m doing is addressing brains. Systems of neurons and emotions. Whether that is real or not… if you feel these emotions, they are real.”

The discussion began to heat up, as some of the best Brainwave events at the Rubin often do.

Schoonover admitted that there are some limits in exploring subjective emotional experience through current scientific tehniques. Speaking to Girard, he said, “I feel that you, and artists in general, are way ahead of the scientists, in that you have figured out ways to create these images, create these feelings. And the science is very impoverished right now, partly for technical reasons. If you want to study the human brain while it’s doing stuff, you have to put it in this fMRI machine. Try to experience the bliss of Parsifal inside there, after someone injects a tracer into your arm.”

Girard spoke about how he engages audiences in his work– the need to build a rich emotional backstory to a character, and then to put a lid on it, restraining performances at every step, asking the viewer to fill in suggested emotions that the characters have come very close to evoking, but have held back from stating obviously, leaving enough room for us to bring our own projections to the viewing experience. Girard’s most penetrating observations came near to the end of the talk: “The full reality of a piece of art is in the blanks,” he observed. “The writer will give you fragments of reality. What a film is really about is the reconstruction of it in the mind of the audience.”

Schoonover was sharp with his questions and offerings from the realm of neuroscience, quick to suggest a relevant insight to the discussion at hand, such as mirror neurons (though he was careful to offer plenty of disclaimers about the limited explanatory reach of such lines of research), and Girard displayed a depth of philosophical consideration of his work, and a genuine interest in what the science can say about perception and emotion. The dialogue proved to be an auspicious start to this year’s series at the Rubin.

Upcoming Brainwave events:

Wed, Feb 27
7:00 p.m.
$20 | Buy Tickets
The Screenwriter | Learn More
Dustin Lance Black (Milk, J. Edgar) + neurobiologist Tom Carew
Sat, Mar 2
3:00 p.m.
$20 | Buy Tickets
The Photographer | Learn More
Mary Ellen Mark + neuropsychologist Daniel L. Schacter
Sun, Mar 3
6:00 p.m.
$25 | Buy Tickets
The New Yorker Cartoonists | Learn More
David Sipress, Paul Noth, Zachary Kanin + neuroscientist Richard Restak
Wed, Mar 6
7:00 p.m.

$35 | SOLD OUT
The Humorist | Learn More
Fran Lebowitz + experimental psychologist Steven Pinker
Sun, Mar 17
6:00 p.m.
$80 | Buy Tickets
The Memorist (The Memory Palace) | Learn More
Nelson Dellis + neuropsychologist Lila Davachi
Wed, Mar 20
7:00 p.m.
$80 | Buy Tickets
The Memorist (The Memory Palace) | Learn More
Nelson Dellis + neuropsychologist Todd Gureckis
Wed, Apr 3
7:00 p.m.
$35 | Buy Tickets
The Virtuoso | Learn More
Zakir Hussain + neuroscientist Seth Horowitz
Sunday, Apr 7
6:00 p.m.
$20 | Buy Tickets
The Lighting Designer | Learn More
Jules Fisher + neuroscientist Margaret Livingstone
Wed, Apr 10
7 p.m.
$20 | Buy Tickets
The Spice Master Learn More
Lior Lev Sercarz + neuroscientist Donald A. Wilson
Wed, Apr 17
7:00 p.m.
$20 | Buy Tickets
The Chef | Learn More
Wylie Dufresne of WD-50
Sat, Apr 20
3:00 p.m.
$20 | Buy Tickets
The Congressman | Learn More
Tim Ryan + psychologist Tracy Dennis, after a screening of Changing Minds at Concord High School
Fri, Apr 26
7:00 p.m.
$25 | Buy Tickets
The Architect | Learn More
Bjarke Ingels + neuroscientist Tom Albright
Page 1 of 6123456