In search of ourselves: Uncovering the secrets of our complex brains
As told by Winrich Freiwald

As in 2024 we celebrate the 300th birthday of the Enlightenment philosopher Immanuel Kant, I am reminded of the great influence he and the Enlightenment philosophy have had on my life. My father was a great admirer of his Prussian compatriot, and so I was exposed early. My upbringing was shaped by Kant’s motto “Habe Mut, dich deines eigenen Verstandes zu bedienen” (sapere aude or dare to think). Thus the existence as a scientist to me really only makes sense in a free and open society. Kant’s epistemology, finally, has been a great influence on my own thinking and interest in how the brain uses incoming signals to re-construct a model of the outside world.

First inspirations
I was born in the city of Oldenburg (Lower Saxony), Federal Republic of Germany, on November 22nd, 1968, less than a quarter century after the Shoah. Growing up in Western Germany at that time, meant enjoying the freedoms of an open society, freedoms the country knew it had not won itself. It also meant growing up with the question of how it was possible that the National Socialists had come to power, a question of particular relevance at the place where they had won their first parliamentary majority in a 1932 regional election, and how it was possible for human beings, and in particular the group of humans I belonged to, Germans, to conceptualize and execute the systematic, industrial murder of other human beings, a question that has haunted me my entire life.

Our core family consisted of my mother, Heide Freiwald, née Behmann, a teacher, my father, Helmut Freiwald, a professor of politics at Oldenburg University, my grandmother (‘Groma’), Hildegard Behmann, and my older sister Anke. My grandmother, whose interests spanned the social and natural sciences, spent countless hours reading to and discussing with her sickly grandson, and was the first influence to develop my interest in science. I therefore could never understand the idea of a gender difference in scientific aptitude. My best friend, Friedhelm von Mering, who for four years lived right across the street, and I shared an early interest in animals and were in constant friendly competition over who knew more animals and more about them. And my father, a political scientist and contemporary historian, but really a polymath, instilled an understanding of the importance of knowledge, of reason, and a sense of the ethos of a professor in me – a sense greatly amplified by the many professors from Toruń, Poland, who were visiting as part of an inter-university exchange program my father had been instrumental to originate. My grandmother and mother helped to fill it with friendship and love, for example by sending food packages especially during the period of martial law in Poland.

"My grandmother and was the first influence to develop my interest in science."

Entering nearby Dietrichsfeld elementary school, I was lucky to have Else Oppermann as my motherly head teacher the first two years, as she was understanding of my many and long disease-related periods of absence and, when I was in attendance, would use me, as she would later recall, as an assistant teacher to help the other students. Principal Heinz Mühlenstedt, our head teacher in years three and four, impressed with systematic explorations of geography, geology, local history, and, on occasion, astronomy using many of the display boards and props held in our school’s secretive stowage.

High School: A place of boundless intellectual curiosity
My intellectually formative years took place at the Altes Gymnasium in Oldenburg, a high school looking back onto a now 450-year history (and maybe more), located directly outside Oldenburg’s historic city center, including the former Grand Duke’s castle, and next to the state theater. It had brought forth two of the great minds of the twentieth century, the existentialist philosopher Karl Jaspers and the Lutheran theologian Rudolf Bultmann. We were encouraged to look up to them, literally, as their portrait pictures were hung high next to the main entrance of the school’s assembly hall.

With its origins as a Latin school, classes naturally included languages, but also the social and natural sciences, all taught at the highest level and, in later years, some conceptually advanced at university level. While we were encouraged towards academic excellence, we were even more encouraged to think independently. I owe everything to the Altes Gymnasium, and I attribute its atmosphere of boundless intellectual curiosity and freedom of exploration to the school’s principal, Dietmar Jungehülsing. I am grateful to many of the teachers (more than I can mention here): Albrecht Sellen, Heiner Fangmann, and Jürgen Weichardt for not only teaching history, but how to think about it; Bernard Jakubuv, for deep introductions into theories of society; to Dr. Karl-Josef Burkhardt for a deeply moving event 40 years after the liberation of the Auschwitz annihilation camp; to Jürgen Weichardt and Dietmar Jungehülsing for taking us on a trip to Southern Poland, including a harrowing visit to Auschwitz; to Dr. Jürgen Ehlert and Hanstheo Weber for changing my conception of theology; and to Dietmar Barke, Gerhard Linkerhägner, Dr. Dietmar Raschke, and Hanstheo Mader for deep and formative introductions into mathematics, physics, chemistry, and biology, respectively, and scientific thinking. Yet, it was by no means the teachers only, but many of my classmates with interests, abilities, and passions different from my own who created new outlets for intellectual exploration and creative expression – for example multiple theater companies – who made this such a stimulating time and environment. There was nothing, it seemed, that we could not do, if we put our minds to it.

"While we were encouraged towards academic excellence, we were even more encouraged to think independently."

It was on this background that, around tenth grade, my interests in the sciences and philosophy merged and brought about the questions and themes that have remained at the center of my scientific thinking ever since. With almost simultaneous exposure to classical mechanics, organic chemistry, and genetics, I was almost forced to wonder about the nature of life and its relationship to the mind – I encountered the ontological mind-body problem. Interested in sub-nuclear- and astrophysics, at a popular science level, I would discover the wonders of quantum mechanics, in particular the Copenhagen interpretation, and with it epistemology, or how it is that we can know anything. Carl Friedrich von Weizsäcker’s writings, bringing several of these themes together in a critical realism, had probably the strongest impact on me at the time. And these two main questions, the mind-body problem and epistemology, ignited my interest in the brain, and, ultimately, the decision to become a neuroscientist. Neuroscience, it seemed to me, was a field of science in which, unlike physics, the fundamental discoveries yet had to be made, and where, in their pursuit, critical medical advances could be helped.

Undergraduate studies: Göttingen
So, the decision was made that I was going to become a neuroscientist. I read widely, including undergraduate textbooks, and contacted leading German neuroscientists to figure out how and where best to study. And, right after high school graduation, I attended my first neuroscience conference in 1988, thanks to Oldenburg University’s Reto Weiler, who had organized it locally and kindly allowed me to listen to a week of talks on a very specific subject, the inner retina. I was most impressed, not surprisingly given my mostly theoretical interests in neurobiology at the time, by Frank Werblin’s talk and the simulation of the salamander retina he showed.

I started my undergraduate studies at the Georg-August-Universität Göttingen. Once a leading center of science until the Nazi takeover, Göttingen in the late 1980s was still a remarkable place for science, not least through the Max-Planck-Institute for Biophysical Chemistry, situated on the Faßberg, and towering over the city. Its first impact on me was by coincidence: just as I moved into a small room in a private home in Göttingen, so did a young Austrian graduate student to work with Nobel Prize winning Max-Planck director Manfred Eigen on evolutionary theory. This student, Martin Nowak, who would later become a leading theoretician of the evolution of social behavior, was a great influence on me and introduced me to mathematical approaches in different fields of biology. We have stayed friends ever since.

Göttingen was also a great place for neuroscience with Erwin Neher, who everyone expected to win the Nobel Prize, and with Otto Creutzfeldt, the doyen of German neuroscience, both directors at said Max-Planck Institute. Yet during my first semester, while I was determined to take as many classes in neuroscience as I could, all but one were canceled due to lack of student interest. But the remaining one, a lecture series on the cerebral cortex organized by Otto Creutzfeldt’s group, highlighting the many different approaches used to unravel its functions, left a deep impression on me; in particular Otto Creutzfeldt’s talk on the mind-body problem (greatly shaped by Kant’s philosophy). I was lucky to be admitted after a personal interview as an intern into his group for the next summer. Maybe my future as an electrophysiologist was determined then, as a late-night experiment of recordings from the retina by Barry Lee turned out to be one of the most exciting things I had ever seen.

"Maybe my future as an electrophysiologist was determined then."

In subsequent semesters, I had more luck with neuroscience courses actually being held, and it brought about some fundamental questions, like how to describe a system as complex as the brain: Annette Zippelius, a theoretical physicist working on spin-glass models, argued that only great simplification would allow us to understand the brain’s functions, while Arthur Konnerth who had just figured out how to conduct patch-clamp recordings from live cells in a slice preparation, emphasized that without detailed understanding of the properties of each neuron type, there was no hope to understand the brain. While these were exciting problems to ponder, my interests in computational neuroscience and vertebrate neuroscience lead me to apply for a transfer to the Eberhard Karls Universität Tübingen in the middle of my undergraduate studies, which was luckily granted.

Undergraduate studies: Tübingen
Tübingen, which quickly became my Swabian home, was a most exciting place for neuroscience. Dezső Varjú and his group offered a systems-theory oriented perspective on neural systems, and Hans-Ulrich Schnitzler’s group offered training in a wide variety of techniques, most notably during a one-semester hands-on course conceived and run by Joachim Ostwald. Once I had transferred, I learned about the exciting program Niels Birbaumer had established in physiological and clinical psychology, and I was immediately drawn to it. I took as many classes as I could, ranging from mathematics to philosophy. And here the Max-Planck Institutes for Biological Cybernetics and for Developmental Biology were closely integrated with university activities offering courses, one run by Axel Borst and Martin Egelhaaf leaving a particularly strong impression on me, and exciting discussions of fundamental questions in neuroscience, e.g., on whether cortical organization was random, as Valentino Braitenberg and Almut Schüz argued for, or organized in a precise manner, as Jürgen Bolz, who had just returned from his postdoc with Torsten Wiesel at The Rockefeller University, proposed. And this was just one pointer towards my current scientific home. Tobias Bonhoeffer, a star student, had just left for his postdoc with Amiram Grinvald at Rockefeller, and Jo Ostwald told us about this amazing place of science, Rockefeller University, that was otherwise largely unknown in Germany.

There were pointers to my future scientific subjects, too. Just days after I had arrived in Tübingen, David Perrett, gave a talk that amazed everyone: it was about cells in the monkey visual system responding selectively to faces. How was it possible that there was something so meaningful as a cell responding selectively to faces – and doing so in just as automatic a fashion as orientation tuned cells in primary visual cortex? But I also remember my impression, which I did not dare share with anyone, that this was a fascinating observation, but that there was probably no hope to uncover a mechanism for the cell’s remarkable properties. The other pointer came from Hermann Wagner who, as co-organizer of a student seminar, suggested a presentation on different theories of the neural code – from holistic processing to so-called “grandmother cells”, the hypothetical neurons Jerry Lettvin had envisioned, that would respond to one’s own grandmother and no one else. This was the topic I happily picked up to present and got me thinking about many of the subjects that would, years later, become the focus of my work.

Doctoral work with Wolf Singer
The question of the neural code was what I really wanted to study. In figuring out which direction to go, I was greatly aided by the German Neuroscience Meeting, which took place in Göttingen annually. I was blown away by the maps of orientation tuning in primary visual cortex that Tobias Bonhoeffer presented with a new imaging technique, and the discovery of their organization around pinwheels. But I was most fascinated by talks about a new theory of neural coding.

The temporal binding hypothesis posited that visual objects were represented by ensembles of neurons were flexibly formed through the synchronous firing of constituent neurons at a timescale of just a few milliseconds. If temporally precise synchronicity defines ensemble membership, multiple cell assemblies could co-exist within the same neural substrate, and thus multiple objects could be encoded separately. The theory, proposed by Christoph von der Malsburg and Wolf Singer, was elegant, it combined my interests in dynamical systems and neuroscience, and, supported by experimental results from the visual system, promised a breakthrough in our understanding of how the brain works in general. So, this was the direction I wanted to go, and I was lucky to be accepted by Wolf Singer to perform my diploma and doctoral work in his group at the Max-Planck-Institute for Brain Research in Frankfurt/Main.

Wolf Singer had created, and was leading, a research group encompassing multiple perspectives on the brain. The largest effort during my time was the use of multi-electrode recordings to test the temporal binding hypothesis, but the group also investigated the functional organization of the visual system and its development, mechanisms of synaptic plasticity, and perceptual psychophysics. Even at the time, such a broad effort to uncover principles of brain function was rare and was possible only because of Wolf Singer’s vast knowledge and deep conceptual thinking, in which these different perspectives were all integrated. Paired with his charisma and infectious optimism, we all shared a sense of advancing science in important ways, and I suspect, all strived to be like him. (At some point more than half the male group members were wearing Wolf Singer’s signature black turtleneck. We all failed, regardless, in this latter pursuit). But I believe we all learned how to think big and pursue a vision unafraid. At least I can see this in many of his former lab members, most notably and admirably in my friends Christian Hansel and Michael Brecht.

"I implemented a new recording technique for the lab to test the hypothesis at the single-cell level."

My interest, motivated by the temporal binding hypothesis, squarely focused on how objects are represented in the brain, as the closest approximation, I thought at the time, of how symbolic representations could be formed. I implemented a new recording technique for the lab to test the hypothesis at the single-cell level. What had been puzzling me though, was that the hypothesis, true in its Hebbian form, would work with a single cortical representation of elementary features: as long as local feature detectors could be bound into a coherent cell assembly, external objects could be represented. Yet, a central feature of the visual system was that it consisted of a multitude of visual areas and, most importantly for my thinking, contained a hierarchy in which increasingly complex properties were represented – such that one could observe face cells, whose critical properties, as the pioneering work of Keiji Tanaka showed, could be reduced, but were more complex than those in the early visual system. I therefore shifted my interest to inferotemporal cortex and investigating the basic properties of the temporal binding hypothesis there. Focusing this approach to faces as a particularly suitable stimulus group to uncover general principles of object recognition, became an important consideration towards the end of my Ph.D. It was something Rainer Goebel, who had built an impressive research program within Wolf Singer’s group around fMRI, and I discussed – using this novel approach to localize face areas within the macaque shape selective cortex. Yet, specific technical equipment was needed to make this approach viable, and so, as my time in Frankfurt was about to end, this had to be left for another time.

Postdoctoral studies with Nancy Kanwisher
Now I was interested in the interplay of vision and attention in the formation of object representations and their selection for sensory and cognitive processing driving behavior, particularly object-based attention, i.e., the phenomenon that attentional selection was shaped by features already grouped into object representations. I had been made aware of this form of attention through researchers at the Cuban Neuroscience Center in Havana; in particular the endlessly creative Mitchel Valdes-Sosa, who were part of a Human Frontier Science Program collaboration that I participated in. I was lucky to be able to pursue this line of research into the neural mechanisms of attention with Valia Rodriguez, also from the Cuban Neuroscience Center, and two highly gifted students, Aurel Wannig and Heiko Stemman, at Bremen University where I had moved from Frankfurt together with Andreas Kreiter when he became a professor there.

While my initial efforts were focused on the impact of attention on population codes in sensory cortex and on object representations, I became increasingly fascinated by the questions of where those top-down control signals originated from and how they would find the right neurons in sensory cortex to enhance. Again, fMRI came into focus, as it had become such a powerful tool to localize functions, both sensory and cognitive ones, in humans. The researcher who drove these efforts most forcefully, creatively, and elegantly in my mind, was Nancy Kanwisher at the Massachusetts Institute of Technology (MIT). In order to learn fMRI, I approached Nancy and shortly after joined her laboratory as a postdoc.

Nancy’s laboratory was vibrant with exceptional students and postdocs bringing cognitive science approaches to the domain of brain imaging and making a rapid number of astonishing discoveries. Since I wanted to learn fMRI for later use in the macaque monkey model and combine it with targeted electrophysiological recordings, I contacted Wim Vanduffel, who I knew from our time as graduate students in Frankfurt, who was working on macaque fMRI both in Leuven, Belgium, and at Mass General Hospital (MGH) in Boston. Wim’s work was part of a larger collaboration driven by Guy Orban in Leuven and several researchers at the MGH including Roger Tootell. They had made highly impressive strides to make a clinical scanner work for investigations in macaque monkeys, driven chiefly by MRI physics and technology developments by Larry Wald, Joe Mandeville, and others at MGH. Wim, at the time, was in Leuven and put me in touch with a student, Doris Tsao, who had learned the approach during a stay in Leuven and brought it to MGH for investigations of the functional organization of the visual system with a particular interest in stereoscopic depth representations. Faces, a major interest in Nancy Kanwisher’s lab after her discovery of the fusiform face area, and still a major focus of research in her laboratory during my time there, seemed a particularly interesting and promising target to both of us, and so we teamed up to try and find a face area in the macaque monkey brain.

"Nancy’s laboratory was vibrant with exceptional students and postdocs."

The discovery of the face-processing system
Thanks to the technical support at MGH and many late hours of scanning, we did find a face area. I was convinced then that it had to contain a very high fraction of face cells. Previous recordings from David Perrett had shown local clusters of face cells, but what we had found was a larger-scale organization that, so I thought not quite logically, we could only see with fMRI because so many face selective cells drove it. Margaret Livingstone, then Doris’s graduate advisor, and a pioneering vision researcher with a long track record of seminal discoveries, kindly offered us space in her laboratory at Harvard Medical School to target this fMRI-identified face region for electrophysiological recordings. This also placed us close to David Hubel, who together with Torsten Wiesel (since my arrival at The Rockefeller University my infinitely wise mentor), had conducted the legendary and Nobel-Prize winning work on development, organization, and function of early visual cortex. And this, I believe, greatly impacted our approach.

Moving the electrode through the brain along a long recording track, traversing cortical regions with many different functions, was exciting in itself, but the critical test, we knew, would come when we crossed the superior temporal sulcus and the electrode tip entered the ventral bank in, if we had targeted correctly, what would be our face area. Indeed, we did observe clear visual responses, audible through the speaker we would use to listen to the signals, which had been absent in the areas we had recorded from before. And with time we realized that the first cell we recorded from was indeed face selective. We advanced the electrode by a small amount to find the next cell. It was, again, clearly visually driven – and also responded more to faces than to non-face objects. Repeating this process four, five more times and encountering what seemed to be face-selective cells, we finally dared exclaim that all the cells we had encountered in that session were face selective. This was too good to be true, and it took as many more such sessions with differing recording trajectories into the face patch to really convince us of this – the fMRI-identified face area was packed, almost exclusively, with face-selective cells. The idea that the human face area was a functional module for processing faces and only faces had received support at the single-cell level. Along the way, we were happy to show these cells to David and Marge. And like David and Torsten, we spent a lot of time recording from individual cells, testing their properties in as many ways as we could imagine, far beyond the few dry metrics that ended up in our papers.

"We finally dared exclaim that all the cells we had encountered in that session were face selective."

We would later, with improved imaging technology and through the integration of multiple techniques, show that there are multiple face areas in the temporal lobe and in prefrontal cortex and that these were connected to form a network – this shown through painstaking experiments by an amazing graduate student, Sebastian Moeller. We also found that the different face areas, while all packed with face cells, would represent faces differently in ways that could be explained by a processing hierarchy that transformed an early image based on representation of faces, into one extracting identity, and we found evidence for local feature selectivity through ramp-shape tuning curves forming a face space, interacting with a holistic representation of the entire face. What was at the time most striking, and still is in hindsight, was the clarity of these findings. And it was the kind of clarity that inspired a sense of beauty and renewed wonder. It thus had turned out that at the seemingly most unlikely of places, the systems processing a highly complex social stimulus, basic principles of high-level object processing and of large-scale functional brain organization, could be revealed with a novel approach.

A new area for attention control
The latter point became most striking to me, when Heiko Stemmann and I, now finally applying fMRI to the study of attention, discovered a new area for attention control. This was maybe more surprising a finding than that of the face areas, since there really was no reason to expect an additional attentional control area at the location where we found it – in the temporal lobe right between two face areas. And even though its properties were fundamentally different from those of face areas – cells were minimally selective for shape, if at all, highly predictive of attentional direction, and if artificially activated, directed the focus of attention in precisely the way expected for attentional control. The first recording of a cell from this fMRI-identified attention control region was just as exciting and joyous as that of the first face cells. The cell was so predictive of where the subject was paying attention to, even predictive of future errors, that Heiko and I played this game where one of us would close his eyes, listen to the cell’s response on the audio monitor and say where the subject was paying attention to. As Ilaria Sani, an outstanding postdoc in my laboratory at the Rockefeller University, would later show, the area was, unlike face areas, connected to two areas in parietal and prefrontal cortex – the clarity of functional specialization and organization was very similar.

"The first recording of a cell from this fMRI-identified attention control region was just as exciting and joyous as that of the first face cells."

Science at The Rockefeller University
Getting the opportunity to start my own group at The Rockefeller University, in the midst of some of the greatest minds of science, was an honor simply unbelievable to me and the source of new inspiration. We set out to work out the neural mechanisms of face recognition at greater depth – identifying genetic underpinnings, cell types, local circuits, and network dynamics shaping neural computations - and to uncover the computational principles the face-processing system implements. But I also came to Rockefeller looking at faces in a new way.

Faces, I had come to realize, were not merely there as passive tableaus for an observer to extract important information from, but they are active agents that can reach us deeply, shaping our emotions, directing our attention, and activating our memories in an automatic fashion. Thus faces offer a unique opportunity to uncover the neural circuits and mechanisms of these complex functions, and the face-processing network can be thought of as a visual front-end of the social brain. We have since used the multi-level circuit dissection approach and made discoveries that have widened and deepened our understanding of face processing and social cognition.

One fundamental question Sofia Landi, an amazing graduate student in the laboratory, and I were particularly interested in was the role of personal familiarity. The face areas we had discovered before appeared to encode face properties without direct experience with the faces displayed in the stimuli we would show. Yet, a major function of face processing is to recognize individuals we personally know, thus linking the perceptual process of visual analysis to memory of people. Sofia would carefully design stimuli of personally familiar, visually familiar (through repetition), and unfamiliar faces, and show that there are two additional face areas in the temporal lobe selective for personally familiar faces and implicating these areas in the process of recognition. Most interesting to us was that one of these face familiarity areas is located in a mysterious, primate-specific part of the brain: the temporal pole, which might hold the key to our sophisticated systems of our social world. The discovery opens the possibility to use faces as a probe to uncover processes of memory formation, organization, and utilization in social cognition.

"Faces play and will play a central role in my work, as they are a key factor in our primate sociality."

The hierarchical circuit-dissection approach we have brought to understand our complex social brains, as, in many ways, allowed me to come back to the roots: to the key question that got me into neuroscience in the first place and to the use of this knowledge in order to improve the human condition. The central question was how brains enable thinking. This question, I believe, has now come into reach. We are pursuing it with paradigms identifying how neural circuits can generate compositional or language-like representations of symbolic information and the programs that operate on them. While this pursuit is abstract, in other work we aim to uncover how faces elicit emotional responses, how emotions are then expressed in the face, and really understand what emotions are in neural terms. We do this in order to reach a better understanding of depression, one of the most prevalent and debilitating syndromes in modern society. I am lucky to be able to pursue these new and challenging directions with a group of highly dedicated and gifted young scientists.

Faces play and will play a central role in my work, as they are a key factor in our primate sociality. Looking back, I wonder whether I found faces, or faces found me, whether I have had a particular interest in them all along. After all a Peruvian artist, Till Freiwald, with whom I not only share my last name, but who is also my cousin, has made it his mission in life to study faces and the people “behind” them, as an artist, creating amazing, literally larger-than-life watercolor studies.

"I am reminded that, as scientists, we place ourselves in the pursuit of a higher cause."

Life
Life is not just for studying, but for living. The year 2024 for me marks six years of marriage to Victoria Yoffie, a mental health counselor, who is taking me on new adventures and discoveries and has given my life new meaning; four years of the birth of my twin daughters, Friederike and Raphaela, who have been teaching me the most profound lessons on life, the challenges of face recognition, and the sense of awe and wonder as we look at the world as if for the first time; and the first year after my father’s death. Life is here, and it is now, it is not just the thing we study. In receiving the 2024 Kavli Prize in Neuroscience, I am reminded that, as scientists, we place ourselves in the pursuit of a higher cause, one that has existed long before us and will, hopefully, outlast us after we are gone. It is at once a humbling and a happy thought. In looking back on my life in science and thinking of the many people who have made it possible and walked with me for some time, and I am grateful for that. As I have made this contribution to advance our knowledge of the world, the large questions, as Immanuel Kant reminds us, will remain without answer. This knowledge, I hope, will renew the sense of wonder that has drawn us to the pursuit of science in the first place.