Press "Enter" to skip to content

The Red Queen Trap

The Red Queen Trap is to be found in the famous Red Queen paradox from Lewis Carroll’s Through the Looking Glass. In this story, a sequel to Alice’s Adventures in Wonderland, Alice climbs into a mirror and enters a world in which everything is reversed. There, she encounters the Red Queen who explains to her the rules of the world resembling a game of chess. Among other things, the Red Queen tells Alice:

It takes all the running you can do, to keep in the same place.

On the face of it, this is an absurd paradox, but it reveals an important insight about a critical point in the life of every system. Let me explain.

Every system, be that a single entity or a large organization, must perform itself into existence from moment to moment. If it stops doing that it succumbs to entropy and falls apart. Spoiler alert, in the long run entropy always wins.

To perform itself into existence every system must expand a certain amount of energy, which is a function of the relationship between its internal state and the external conditions it operates in. In other words, it must expand some energy on keeping its internals working smoothly together, and then expand some energy on resisting and adapting to adverse external conditions.

The better adapted a system’s internal state is to its external conditions, the less energy it must dedicate to perform itself into existence, and the larger the potential energy surplus it can use to grow, expand, or replicate itself.

However, external reality is complicated [not to be confused with complex] and changes dynamically in ways which cannot be modeled over the long term and require constant adjustments by the systems [organisms, humans, organizations] operating within it. In other words, an external state observable at time A is no longer present at time B.

This is a problem for all systems, because it requires them to change how they operate.

It is a small problem for simple systems which are usually internally homogeneous and highly distributed. Their homogeneity means they don’t need to spend much energy to maintain their internal state, and their distributed topology means they make decisions and react very fast.  

It is a serious problem for complex systems [large organizations] which are usually rather centralized and heterogeneous. Their heterogeneity means they must expand a lot of energy to maintain a coherent internal state consisting of various qualitatively different elements, and their centralized topology means they react and make decisions rather slow.

It is a profound problem for complex hierarchical systems [large organizations with vertically integrated decision making] which consist of multiple heterogeneous elements stacked along one or more vertical axes. Vertical integration means that each successive layer going up is further removed from direct exposure to external conditions and is therefore slower in adjusting to them.

A system might be quite successful in adjusting its internal state to external conditions at time A, but a later time B might present a different configuration of conditions to which the internal state of the system at time A is profoundly inadequate. The more complex the system, the more energy it must expand in adjusting to changes in external conditions from time A to time B.

Complex hierarchical systems have the hardest time in making these adjustments because key strategic elements of their internal state [i.e. decision-making centers high in the hierarchy] are far removed from direct contact with external conditions. To orient themselves and perform the system’s OODA loop they rely on communication about external conditions reaching them from the periphery of the system, while orders on necessary adjustments must travel the other way, from center to periphery. This takes time, and the more layers the signal communicated from the periphery must pass through on its way to the center the more abstracted it becomes from external conditions. In other words, the center receives a highly imperfect version of the external conditions about which it must make adaptive decisions.

Over time, this generates a growing number of errors in the internal state of the system, requiring more and more energy to be routed to internal maintenance [i.e. bureaucratic paperwork], leaving less and less surplus energy for adaptation, growth and expansion. Eventually, and this stage can arrive very fast, the system reaches a state of pseudo-equilibrium in which all energy it can produce goes towards internal maintenance and there is zero surplus energy left. This is where the Red Queen Trap kicks in:

The system does all the running it can do, to keep in the same place.

How does the trap work? First, from the inside everything in the system still seems to be operating smoothly and things are humming along in accordance with external conditions at present time A. However, this is a false perception of equilibrium, because when external conditions invariably change in future time B the system will have no surplus energy reserves to adjust to the new conditions.

The more imperfect the version of external conditions reaching the center of decision-making, the more pronounced the system’s inertia in this state of pseudo-equilibrium, and the deeper it goes into the Red Queen Trap.

Second, having eventually discovered there are no more surplus energy reserves left, the system must now make a choice.  In the absence of surplus energy and provided there is no energy transfer from the outside, it must somehow free up energy from within its internal state in order to adapt. The question is, which internal elements should be sacrificed to free up that energy? This is where the Red Queen Trap’s simple elegance is fully revealed.

Essentially, there are two options – a seductively easy one and an unthinkable one. The seductively easy option is to sacrifice the periphery, or elements of it, and preserve the decision-making center. It is an easy choice for the center to make because it naturally sees itself as the key element of the system and this choice allows it to remain intact. It is a seductive choice because the center suddenly finds itself with a flush of spare energy which it can use to maintain the pseudo-equilibrium and often even to grow itself at the cost of the periphery. Alas, the elegance of the trap is in the fact that the seductively easy option removes the center even further from external conditions; less periphery equals less opportunities to observe and react quickly to external reality, thereby further magnifying the initial conditions that brought the system to this state in the first place. By making that choice the center sinks further into the trap.

By contrast, the unthinkable option is to sacrifice the center and preserve the periphery, thereby flattening the internal structure of the system into a less hierarchical form. It is an unthinkable option for the center to make because, as pointed out above, it naturally sees itself as the key element of the system and this choice forces it to sacrifice itself. It is also unthinkable because it involves a thorough rethinking of the internal structure of the system, which until that moment was organized entirely around vertically integrated decision making, with little to no autonomy in the periphery. The centre must not only sacrifice some of itself, but also reorganize the periphery in such a way so that it can now perform those functions in place of the center. This would allow the system to free itself from the trap.

Obviously, most systems choose the seductively easy option and the Red Queen Trap eventually grinds them into oblivion. Those few systems that go for the unthinkable option escape the trap and, if they remain persistent in their application of the unthinkable, learn how to go different places with running to spare.

2 Comments

  1. Really enjoyed this. Resonates a lot.

    But I wonder about whether the two solutions you present are the only ones, both abstractly and in practice?

    Arguably, the two options are addressing the centralised/distributed axis you use to characterise systems (the other being homogeneity). Though the “sacrifice the periphery” does end up increasing homogenity, I’d argue it tends to be driven more by centralisation. The “sacrifice the center” approach decreases centralisation.

    I wonder about homogeneity.

    What I see in Oz higher education around learning and teaching is attempts to increase consistency i.e. homogenity. We all use the same systems. Measured against the same standards. Use the same process. As you indicate, the argument for such moves is that it is more efficient. Where does this fit? Does it?

    Personally, I don’t think it will be a successful solution. It papers over the the underlying diversity. It also seems to tend to increase centralisation. As the standards are controlled from the centre, not the periphery.

  2. Ted Ted

    Thanks David, these are good points.

    You can view the two options I identify as the two distilled poles along an axis. In most cases there are going to be degrees of overlap between the two as an organisation struggles with the trap.

    Homogeneity/heterogeneity is a key quality of systems. I do not explain it here as the essay is too abstract already, but I understand it as the degree of qualitative difference between the internal elements of a system. These could be different units in an army, or different bureaucratic units in an organisation, or ethnic groups in a community. The point is that the more heterogeneous a system is, the more energy it has to spend on maintaining internal coherence.

    You are correct that sacrificing the periphery increases centralization – suddenly the system is left with less periphery and more center. It does not however automatically lead to an increase in homogeneity. Decision-making layers in a hierarchical system are usually highly specialized, or develop quasi-specialization as they try to differentiate themselves from other decision-making layers in the competition for energy within the organization. In effect, when viewed from the outside, they all seem to be highly important and indispensable, and creating this effect is their primary occupation. After all, this is what keeps them in the command axis.

    I get your point about Oz higher ed. The thing about the drive for internal consistency [systems, language, policies] is that it is a function of a system’s continuous effort to maintain internal consistency. As I point out in the essay, an internally homogeneous system needs to spend only a minimal amount of energy on maintaining coherence [think of the crude corporate team-building exercises]. A highly heterogeneous system has to spend an enormous amount of energy, and develop a complex set of rules, terminology and practices in an effort to build and maintain internal coherence [a pseudo-homogeneity]. As you point out, this is what Oz academia is trying to do. The thing is that all of these efforts have a huge energy cost, and leave the system ever more vulnerable to changes in the external environment.

    Further centralization delivers the illusion of a quick solution [seductively easy], but is a one-way ticket to further entropy in the Red Queen Trap. The way out of the trap is to do precisely the opposite – remove as many layers from the central axis as possible and delegate their responsibilities to the periphery thereby increasing decentralization. Alas, this is in the realm of the unthinkable for a hierarchical system.

Leave a Reply

Your email address will not be published. Required fields are marked *

one × 3 =

This site uses Akismet to reduce spam. Learn how your comment data is processed.