section 3


Foundations: Laying the Groundwork (7)
Levels of Cognitive Interaction: Variations on a Theme (8)
Representations of Cognitive Interaction : Interpreting the Signs (7)
Summary: Emergence of Mind (6)


foundations 1 of 7

Laying the Groundwork
How do we apply an emergent systems perspective to cognition? As the basis for understanding emergence, levels are the main organizational feature of the emergent systems framework. They’re a way of zooming into phenomena to study patterns and interactions at different scales. With levels, we can start with an overview of cognition and work our way in.

foundations 2 of 7

The idea of levels in models of the brain, emotions, attitudes, etc. is nothing new. Although most of these models do not differentiate between emergent and other kinds of levels, they can still provide a good picture of the territory we are trying to explore.

foundations 3 of 7


Many models in both academic work and popular culture involve three levels of cognition. The most well-known example is probably MacLean’s triune brain, which maps the evolution of various brain structures to certain types of brain functions. Although not considered scientifically valid as an explanation of how the brain actually works, it is a helpful approximation for the organization of cognitive processes and structures.

  • Reptilian Complex: instinctual behaviors, “lizard brain”
  • Limbic System: pro-social behaviors, “mammalian brain”
  • Neocortex: reflective behaviors, “human brain”

Truine Brain | Wikipedia
Rule of Three | Wikipedia

foundations 4 of 7


A similar organization is evident in Ortony, Norman, and Revelle’s model of emotional affect in the context of reactive, routine, and reflective levels of cognition, which is also the basis of Norman’s model of visceral, behavioral, and reflective levels of emotional design. This work, however, doesn’t involve mapping brain structures to brain functions. Instead, the researchers were considering how different levels of cognition represent various aspects of the same information (emotion).

  • Reactive / Visceral: proto-affect; emotional responses based on instinct, physical sensation
  • Routine / Behavioral: affect; emotional responses based on habits, learned behaviors
  • Reflective: emotion; emotional responses based on thought, reasoning

Affect and Proto-Affect in Effective Functioning | Ortony, Normal & Revelle
Three Levels of Processing: Visceral, Behavioral, and Reflective | Norman

foundations 5 of 7


Another useful perspective on levels of cognition involves how we understand reference, or the relationship between what is being communicated (the signifier) and how it is being communicated (the signified). For example, an image of a dog, a paw print, and the letters d-o-g are all signifiers of the idea of dogs. However, the first bears a physical resemblance to a real dog, the second is an intuitive reminder of real dogs, and the third is an abstract form assigned to refer to a dog. In his work on sign theory, Peirce refers to these three types of signs as icons, indices, and symbols.

  • Icon: signifier refers to signified based on physical resemblance (image of a dog and dogs)
  • Index: signifier refers to signified based on co-occurrences (a paw print and dogs)
  • Symbol: signifier refers to signified based on assignment (word “dog” and dogs)

Peirce’s Theory of Signs | Stanford Encyclopedia of Philosophy

foundations 6 of 7


Using sign theory, Deacon models how language evolves from the iconic, indexical, and symbolic reference represented by signs. Rather than looking at various levels of cognitive processing, Deacon focuses on various levels of information (reference) as a function of cognition. Although he doesn’t specifically mention emergent levels, he describes how iconic reference gives rise to indexical reference which gives rise to symbolic reference.

  • Iconic reference: unconscious substitution of signifier (image of a dog) for signified (dogs)
  • Indexical reference: remembered mapping of signifier (paw prints) to signified (dogs)
  • Symbolic reference: learned substitution of signifier (the word “dog”) for signified (dogs) and mapping to other signs (references to / of “dog” and dogs)

The Symbolic Species: Symbols Aren’t Simple | Deacon

foundations 7 of 7


How do these models inform each other? Not only do they focus on different topics, but they also represent different cause and effect relationships, from direct A=B mappings, to linear A-to-B-to-C progressions, to nested (((A) B) C) evolutions. Clearly the models represent specific points of view, but how might they collectively contribute to our overall picture of levels and cognition?

  • Iconic reference: unconscious substitutions based on physical resemblance between signifier and signified
  • Reactive / Visceral: responses based on instinct, physical sensation
  • Reptilian Complex: instinctive behavior, “lizard brain”
  • Indexical reference: remembered mappings based on co-occurrence of signifier and signified
  • Routine / Behavioral: responses based on habits, learned behavior
  • Limbic System: pro-social behavior, “mammalian brain”
  • Symbolic reference: learned networks of substitution and mapping based on assignment of signifier to signified
  • Reflective: responses based on thought, reasoning
  • Neocortex: reflective behavior, “human brain”

levels of cognitive interaction 1 of 8

Variations on a Theme

Despite their differences, models involving levels of brain organization, emotion, signs, and reference suggest consistent underlying themes for each level of cognition. As levels of an emergent system, each theme should relate to a type of interaction. Based on the identified themes, we can organize these interactions into sensory, intuitive, and abstract.

Sensory interactions: instinctive, body, iconic

  • Iconic reference: unconscious substitutions based on physical resemblance between signifier and signified
  • Reactive / Visceral: responses based on instinct, physical sensation
  • Reptilian Complex: instinctive behavior, “lizard brain”

Intuitive interactions: habitual, memory, indexical

  • Indexical reference: remembered mappings based on co-occurrence of signifier and signified
  • Routine / Behavioral: responses based on habits, memory, learned behavior
  • Limbic System: pro-social behavior, “mammalian brain”

Abstract interactions: reflective, thought / language, symbolic

  • Symbolic reference: learned networks of substitution and mapping based on assignment of signifier to signified
  • Reflective: responses based on thought, language, reasoning
  • Neocortex: reflective behavior, “human brain”

levels of cognition interaction 2 of 8


Each level of interaction gives rise to a type of representation. These representations are higher-level integrations of the patterns of interactions from which they emerge. They are similar to the traffic emerging from the interactions of cars slowing down, the music emerging from the interactions of sounds from instruments, the pie emerging from the interactions of the molecules of its ingredients. As derivatives of the level they represent, we will refer to these representations as perceptions, associations, and conceptions.

Perception: integrated representation of sensory-level patterns

  • Sensory interactions: instinctive, body, iconic
  • Iconic reference: unconscious substitutions based on physical resemblance between signifier and signified
  • Reactive / Visceral: responses based on instinct, physical sensation
  • Reptilian Complex: instinctive behavior, “lizard brain”

Association: integrated representation of intuitive-level patterns

  • Intuitive interactions: habitual, memory, indexical
  • Indexical reference: remembered mappings based on co-occurrence of signifier and signified
  • Routine / Behavioral: responses based on habits, memory, learned behavior
  • Limbic System: pro-social behavior, “mammalian brain”

Conception: integrated representation of abstract-level patterns

  • Abstract interactions: reflective, thought / language, symbolic
  • Symbolic reference: learned networks of substitution and mapping based on assignment of signifier to signified
  • Reflective: responses based on thought, language, reasoning
  • Neocortex: reflective behavior, “human brain”

levels of cognitive interaction 3 of 8

There are all sorts of ways to characterize levels of cognition. Comparing existing models with levels of cognition-related topics gives us an overview of prominent themes. Our description of cognitive levels is a summary of these themes as types of interaction processing.

levels of cognitive interaction 4 of 8

The first level of interaction processing is sensory. It involves the interactions of our bodily sensations. Everything we physically experience in the world is processed through this level into perceptions. This is the information we attribute to the body — our ability to see, hear, smell, taste, touch, and move.

levels of cognitive interaction 5 of 8

The second level is intuitive. At this level, we process interactions among perceptions over time and intuitively learn how various perceptions are associated with each other. This information is most often attributed to the heart — our pre-verbal ability to understand relationships.

levels of cognitive interaction 6 of 8

At the third level, there is the processing of abstract interaction. This processing depends on words to represent associative interactions, giving form to conceptions which we can then use and manipulate in the physical world. We generally attribute this information to the mind — our ability to use words to communicate with each other but also with ourselves via our own verbalized thoughts.

levels of cognitive interaction 7 of 8


Photo Credit: jean-louis zimmermann

We can imagine a fourth level as well. However, it is different from the other three in that it does not necessarily give rise to a fully integrated representation. This is a narrative level at which we process the interactions of conceptions into interpretations. It is the information we may attribute to the spirit — our ability to create context and meaning.

levels of cognitive interaction 8 of 8


By themselves, emergent cognition’s sensory, intuitive, abstract, and narrative levels simply refer to types of interaction. Beyond the description of individual levels, there is still the question of how the interactions of perceptions give rise to associations, and the interactions of associations give rise to conceptions. To address this question, we’ll need a better understanding of the nature of representation.

reference & representation 1 of 7

Interpreting the Signs
If cognition is essentially a form of communication, then representations are essentially forms of reference like the signs described by Peirce and Deacon. In his model of signs and reference, Deacon illustrates how one form of reference evolves into another. Using a modified version of that model, we can describe the relationships among types of reference, which can then inform our description of the relationships among types of representation.

reference & representation 2 of 7


Let’s say you know someone named Eduardo. I can show you a picture that looks like Eduardo, play a recording that sounds like Eduardo, or even give you a whiff of something that smells like Eduardo. Any of these may act as a substitute sensory representation of the real Eduardo. The image, recording, and smell correspond to a visual, auditory, or olfactory embodiment of Eduardo’s physical presence. Even if Eduardo isn’t actually there, his physical presence is being mimicked through one of our senses and tricking us into thinking that he is. That’s iconic reference.

reference & representation 3 of 7


Then there’s Maria. Let’s pretend Maria is your best friend. If I play Maria’s favorite song, show you Maria’s favorite outfit, or imitate Maria’s signature dance move, this will act as an associative representation of Maria. The song, outfit, and movements act as triggers that remind you of Maria. She could be a thousand miles away, but these triggers are so closely associated with Maria that you’ll practically expect to see her walk through the door at any minute. That’s indexical reference.

reference & representation 4 of 7


Now let’s imagine that your idol has always been the historical figure, Galileo. Naturally, I feel compelled to tease you about this by repeatedly saying “Galileo”, writing “Galileo” on all your stuff, and even learning to spell “Galileo” in sign language. Your unconscious has learned that these auditory, written, and gestural perceptions are all versions of Galileo’s name — a word his mother decided to use to signify him as a person. So, just like with “apple,” when you hear, read, or see his name, there’s that somewhat confused part of your unconscious which acts like he’s there. Or at least, there in your mind. Nevermind that you haven’t ever actually met him. This index-as-icon transformation is a necessary but not sufficient condition for symbolic reference.

reference & representation 5 of 7


After your unconscious has learned to directly map any physical perception of the word “Galileo” to your imagined perception of Galileo, it doesn’t stop there. Now, every time you have any audio, visual, or gestural perception of “Galileo,” your unconscious automatically has a “Galileo”-as-imagined-perception-of-Galileo moment. Subsequently, its next trick involves processing the relationships from both word and referent (associations, perceptions, and all) into one integrated representation of “Galileo.” That’s symbolic reference.

reference & representation 6 of 7

At some point in this imaginary scenario, I decide to stop teasing you by mindlessly repeating his name. Instead, I might use “Galileo” while explaining what a big nerd you are, or in the text I send you (about what a big nerd you are), or perhaps during my interpretative dance performance of Galileo’s life story, which just so happens to feature the song American Rhapsody by Queen (because it takes one to know one). In any of these contexts, you’re interpreting “Galileo” in relationship to everything else I’m saying, writing, or doing. That’s what we’re calling contextualized symbolic reference.

reference & representation 7 of 7


By exploring types of reference, we begin to get an idea of how iconic reference relates to indexical reference and how both iconic and indexical reference relate to symbolic reference. This, in turn, gives us a starting point for exploring the relationships among forms of representation and the processes by which sensory interactions give rise to intuitive interactions and intuitive interactions give rise to abstract interactions.


representations of cognitive interaction 1 of 7

Icons, indexes, and symbols are forms of reference that exist in the physical world. Representations, on the other hand, are forms that only exist within a cognitive system. We could say that perceptions, associations, and conceptions are basically the icons, indexes, and symbols of cognition.

representations of cognitive interaction 2 of 7

Perceptions are the icons of cognitive representation. While iconic reference involves substituting and/or confusing the physical presence of Eduardo with something physically similar to him, perception involves substituting the physical world with the iconic representation of our experience of the physical world. The result is a kind of virtual reality that is literally all in your head. See that sunset? Well, you’re not actually seeing the sunset. You’re seeing the perception created by your brain processing sensory interactions between your eyes and the light coming from the sunset. It’s worth repeating: Everything you’re consciously aware of experiencing in the physical world is really just in your head. Sure, the real world exists (probably), but perception isn’t a direct interface with the real world, it is only a replication of it.

representations of cognitive interaction 3 of 7

IMG_1715.JPGAssociations are the indexes of cognitive representation. They are derived from the interactions of perceptions. After you see, hear, smell, taste, touch, or otherwise have a perception of one thing, you then intuitively expect to have a perception of something else. Associations will have different strengths depending on your experience. If you’ve only ever perceived red apples, you may more closely associate apples with the color red than, say, a normal person who is familiar with differently colored apples and whose mind isn’t completely blown by trying to imagine that an apple could ever possibly be green.

representations of cognitive interaction 4 of 7

Most associations are clearly differentiated. As in not fully integrated. As in your unconscious can tell one thing apart from another, and treats them as different perceptions. But words are special. With words, we learn to fully integrate the perception of the signifier (the word) and the signified (whatever the word refers to). Our unconscious no longer discriminates between the two and begins to treat them as if they are the same perception. So you’ll hear the sound of the word “apple” and part of your brain acts like it’s seeing a real apple. It’s as if “apple” has become an iconic substitution for the perception of apples.

representations of cognitive interaction 5 of 7

Conceptions are the symbols of cognitive representation. As a prerequisite, your physical perception of a word must be super-associated with an imagined perception of what the word signifies. Then, whenever you perceive that word, your unconscious integrates the associations and perceptions of both into a single representation or conception of the word. This is a dynamic process, so you may represent the same word differently each time you perceive it. Certainly, your understanding of the word changes, too, as you experience new and different perceptions and as you learn new and different associations. So you probably will not always think of Galileo quite the same way that you do now.

representations of cognitive interaction 6 of 7

Interpretations are the contextualized symbols of cognitive representation. This is when we process the interactions of multiple words-as-concepts (like “Galileo”). In addition to processing the words as perceptions and associations, your unconscious maps a constellation among the words-as-concept. Basically, your brain is integrating a network of relationships among all these words-as-perceptions, words-as-associations, and words-as-concepts into a sense of meaning.

representations of cognitive interactions 7 of 7


Icons, indexes, and symbols give us a reference point for describing the interaction processing by which sensations gives rise to perceptions, perceptions gives rise to associations, associations gives rise to conceptions, and conceptions gives rise to interpretations. This description also helps us understand how cognition functions as a kind of internal system of communication. This in turn is the basis of an emergent systems perspective of the relationships among body, sensing, sensation, and perception; memory, feeling, intuition, and association; words, thinking, abstraction, and conception; and language, understanding, narration, and interpretation.

emergent cognition 1 of 6

Emergence of Mind
The physical emergence of matter provides a template for imagining the cognitive emergence of mind. The same way that interactions of simple physical forms give rise to more complex physical forms, the interactions of simpler cognitive representations give rise to more complex cognitive representations. As embodied in the emergent systems framework, this type of emergence can be described in terms of patterns and levels of interaction.

emergent cognition 2 of 6

Cognition as an emergent system involves specific types of levels, patterns, and interactions. Instead of levels of matter (particles, atoms, molecules, etc), there are sensory, intuitive, abstract, and narrative levels. Instead of the laws of physics, there are patterns. Rather than focusing on naturally occurring interactions in the world, the focus is on interactions created, connected, and combined within a cognitive system.

emergent cognition 3 of 6

In both physical and cognitive emergence, patterns of interaction processing effect integrated representations. These representations are themselves interactions, but they function as individual forms at a higher level of processing. Emergent cognition involves perceptual, associative, conceptual, and interpretative forms of representation. Unlike physical representations, however, these cognitive representations only exist and interact within the cognitive system.

emergent cognition 4 of 6

Sensing, feeling, thinking, and understanding each involve different kinds of representation which are effected by different levels of processing. Yet all of them are derivatives of the same fundamental interactions. They reflect forms of meaning rooted in and rising out of our physical experiences in the world.

emergent cognition 5 of 6

Herein lie the essentials of an emergent cognition framework. It is a way of imagining how conception emerges from perception, thinking emerges from sensing, the mental emerges from the physical, mind emerges from body. From this perspective, we have a frame of reference for understanding cognition.

emergent cognition 6 of 6

What changes when we look at cognition through a lens that lets us zoom further out and closer in through multiple points of view? There are some interesting implications. Not so much in terms of what we know about cognition, but how we interpret what we know about cognition. At the very least, there is the hope that someday our lack of vision will no longer result in the cruel and unusual punishment of elephants, metaphorical or otherwise.