Two years of work, one NSF grant, and half a million micro-etched neurons later, Greg Dunn has dunn it again.
Working with Brian Edwards, a physicist at UPenn, the prolific neuroscientist-turned-artist (whom we first interviewed back in 2011) has created Self Reflected, an 8’x12′ piece comprised of smaller, square panels, which altogether displays realistic temporal dynamics of neural activity, seen in dazzling colored reflections on the gold-etched surface when the viewer walks around the piece. The title of the piece speaks to the nature of that neural activity on display, which reflects the sort of activity that would underpin the perception of a piece of art, such as this one.
Self Reflected, which has already been covered by Wired and the Huffington Post, is being called the ‘world’s most complex and detailed artistic depiction of the human brain’ to date. The piece is now on view at the Franklin Institute in Philadelphia.
I spoke to Greg Dunn last week ahead of the opening of Self Reflected.
Noah Hutton: When you decided to start this project, did you know how big the mountain was that you were about to climb?
Greg Dunn: I thought I did. And I think what I thought was the top was really just a foothill of this project. I had thought of this as the comprehensive brain piece that I had been wanting to do for a long time, one that really illustrated all the connectivity and circuit dynamics and stuff like that. But the thing which really kind of distilled it and gave us the opportunity to do it was that Dr. Diane Witt, who had been the neural systems cluster head at the NSF, had seen some of my work at some meetings, and she invited me to exhibit some of my work at the National Science Foundation in Washington.
So Brian Edwards and I had a big exhibit at the NSF featuring a few micro-etchings, and they invited us to come down and give a talk about the work. So we basically had our proof-of-concept experiments on our wall for six months. And they invited us to apply for a grant, because they saw a lot of potential in this science-art as outreach, as being able to inspire a new generation of scientists. So we applied for the EAGER Award, which is a high-risk, high-payoff grant to do this installation at the Franklin Institute, which I think to them was appealing because a concept like the brain is very difficult to communicate through factoids, which is what the typical dissemination to the public is. You know, you can say 100,000 times that the brain has 85 billion neurons, and nobody can really digest that.
So our plan was to make a piece of art that was so grand in its scope that it would touch people emotionally. I really believe that people learn most effectively when their emotions are involved. And there’s data to back that up as well.
NH: You had tried to capture a lot of that complexity in your previous work. In many of the etchings that I had seen and that I had exhibited in that show in Seattle for example, as you walk around the etching, there are already temporal dynamics in that older work. So when you got the NSF grant and you approached this new project, what were the new challenges, what were the new techniques that this funding allowed you to explore here, that ended up making this project so complex?
GD: Yeah, that’s a good question, because I had felt like we had micro-etching slam-dunked, and that we could basically do whatever we want. The etchings that you had seen up to that point had been ones which were separating the surface into very few reflective channels. The etches are symmetrical–they’re wavelike structures, meaning that if you have a light above it or below it the image will be the same. And that dictates how much animation you can fit into a micro-etching. These new pieces, because they’re specifically meant to be animated, are utilizing pretty much all of that, meaning that instead of having five channels now we have subdivisions of degrees acting as channels.
Making one micro-etching, you only really need to be thinking about the lithography conditions for that one etching. You don’t need to worry about how it’s butting up against other etchings. In Self Reflected we now have action potentials which are traversing four or five plates that are butted up against one another. And they essentially have to be perfectly aligned to be able to fool your brain into thinking that it’s one continuous micro-etching that’s conveying the information.
Handling this amount of data and this degree of complexity was a huge new issue. We were way more beholden to the actual anatomy of the brain on this project than I had been in the past. We had selected an oblique sagittal slice and identified all the gray-matter regions, did this deep research as to what neurons appear there, what they look like, what they’re connected to, and made these big databases of that. And so it’s one thing to compile all that data and have it be such that the etching is just randomly sparkling neurons, which really doesn’t tell you anything about what’s actually happening in the brain. We wanted to choreograph the reflectivity so that it was actually emulating what might be happening in your brain when you’re looking at a piece of art similar to the one we’ve made here.
We were able to paint down using different types of techniques, including hand-drawing, scanning into the computer, vectorizing shapes– neural shapes– and turning them into brushes that we could paint down very large regions of neurons. And in fact, the final piece has about a half-a-million neurons. We painted the neurons separate from the axons. So the major axon tracks in this piece were created using Adobe Illustrator through kind of guide images that were taking from diffusion spectrum imaging. So the challenge became that we mathematically reduced each neuron into a point. We then wrote this massive algorithm, and we want certain neurons to connect up with certain axons– which we’ve drawn, which are a separate file– with some amount of chaos in the connections. We want to designate other things like, “Are they going to fire as a burst? Are they going to fire individually?” And what the algorithm is doing is it is placing causal contingencies on everything, routing this information in such a way that it’s trying to emulate the chaos of connectivity of the actual brain. So it’s not just choosing the first thing that’s in its path; it’s choosing things based on these criteria that you’re inputting, which are as close to the kind of anatomy as we could reasonably try to replicate.
NH: Were you looking at one set of data in particular, from one institution? Or were you combining datasets?
GD: Yeah, that’s a good question. We basically scraped it up wherever we could find it. So for a lot of the research in terms of the neuromorphology, we went to primary literature. One thing that I am pretty keen on making sure that people are not under the impression of is that this is some scan that we took of the brain that we just are basically displaying. Like, it’s so ridiculously not that. It’s very much assembled from a ton of different sources to be able to achieve what we have done.
NH: And then back to the more artistic side of it for a second: you already had temporal dynamics as you moved around your previous work. But it’s almost like with this new piece you’re fulfilling the “prophesy,” so to speak, of temporal dynamics. Now those dynamics actually show you something about patterns of activity. Am I gathering that correctly?
GD: Yes. There’s a causality to this piece which was not there in previous pieces. For the most part it was things sparkling, or things separated into channels to be able to kind of demonstrate the technique. But this one takes it much further than that.
NH: As someone who’s trained in neuroscience, when you got into the nitty gritty of this piece, designing the algorithm to make those chaotic connections, did you find yourself encountering things you didn’t know about the brain, just by encountering such complex datasets and having to deal with them?
GD: Oh God, yeah, absolutely. I mean, I definitely came away with this project with a better understanding of how the brain is constructed. And one of the things that immediately comes to mind is how important just the physical layout of the brain is, and how much the neural connections are kind of contingent upon that, and how the brain has wired itself in order to respect where everything is in space, both through development and, you know, in its later orientations. One of the principles I had to learn in grad school is that if, for example, your visual system is trying to interpret something flying across your visual field, there will be neurons which are tuned to certain details at very specific pixel in your visual field. So let’s say you have five neurons that fire in response to something going across your visual field. Those five neurons are then connected to a downstream cell in such a way that as they fire in time, their postsynaptic potentials reach the soma of the receiving cell at the same time in order to increase the threshold electrochemically enough to be able to get that cell to fire an action potential and tell you perceptually that you’ve seen something fly across your visual field.
So that kind of stuff is just everywhere in the brain. Like, it’s easy to imagine that in a visual field; but I think for much more abstract processes consciousness is to a large extent coming from the fact that you have these contingencies in how things are connected up temporally in space, which really just blew my mind, you know, working on this project.
NH: So then when you present this piece in a public space like the Franklin Institute, how much of that information do you want to give the viewer? I mean, are you – what kind of wall text would someone expect?
GD: There’s only a certain amount that’s going to be able to be transmitted. And the most important thing to me initially is that somebody just stand in front of the thing and take a moment to just say, like, “Wow.” You know, everybody who’s looking at that thing has got a brain. The take-home message is that the brain is super-complex; there’s a lot going on in it all the time. And I also wanted to bridge this gap in the popular press as to how the brain is typically illustrated, which is that you’ll either see it, like, on the cover of Scientific American–there’s a kind of transparent person, and you can kind of see their brain.
And then there’s one where you see a field of maybe three or four neurons, which are all 3D rendered. And maybe they have little sparks of light going from here to there. And so people are used to seeing that image. But I think the average person really doesn’t have any idea how a whole ton of neurons makes up the macroscopic brain. I want to change the consciousness of how the average person thinks about the brain with this piece.
NH: When I interviewed you back in 2011, I asked you about how the work changes your own conception of the brain. And we’ve talked again about that a bit today. But you talked back then about how there was this tension between the spontaneity of being a painter, and what’s required when you work with the actual scientific imagery, which kind of forces you to pick out patterns and be a little more analytical. And it seems to me like in this project, it’s been very mathematical, it’s been algorithmic. And in the end it’s a dazzling piece. But I’m wondering how you feel that tension playing out in your own work now, and if you have any urge now to get back to the painterly spontaneity.
GD: Such a perceptive comment, because, yeah– this project is basically all engineering at this point. I mean, a lot of the artistic decisions were made early. There were aesthetic decisions being made along the way. But actually making the etchings themselves is pretty much straight-up engineering. I mean, just having a lockdown on all the variables that are going into it, and knowing how all of them affect it, is super important. With the gilding, for example, I mean we’re talking about, like, hand-making these objects with tolerances within three to five microns.
NH: Oh my gosh.
GD: And then it comes to coating the plates, because we’re then putting the gold leaf on there. So you need to get a super-thin layer of glue over that surface that allows you to put the gold leaf down without filling up the etches, because that’ll just completely destroy the thing that you made. So it was just stressful. And we had to completely remake the thing twice.
But now it’s finished, and the important thing is that the first time that I saw it turned on with the plates put together, I had that emotional reaction of being fooled into thinking that it was one giant cohesive piece. And that’s what it’s all about: You’ve got to have that initial emotional reaction to it.
GD: And I’m sure that it will provide that for a lot of people. And so in that regard I feel like this project, though painful, will have been a success and will be one I think that the Franklin will have up for a long time.
Self Reflected is now on view at the Franklin Institute in Philadelphia. Visit Greg Dunn’s website for more information.