Why Robots Fall Down
by Lawrence Goldfarb, PhD
The following talk was presented as a keynote address to Cybernetics in the Art of Learning, the 1993 American Society for Cybernetics annual conference in Philadelphia. Following the formal paper, I led the group through a twelve-minute standing Awareness Through Movement lesson. After that, Heinz von Foerster and Humberto Maturana gave formal comments on the paper, which were followed by a discussion with the audience.
The talk builds on the link between cybernetics and the question of how human beings coordinate movement, a connection that originally made with the publication of Norbert Wiener's Cybernetics in 1948. In the introduction to his classic text, Wiener talks about the importance of kinesthetic perception for the guidance of movement. He argues that we cannot think about perceptions as inputs and movement as outputs, but rather that we must consider the nervous system as an integrated whole, one where circular causality, in general, and feedback, in particular, govern sensory-motor correlation.
As you will see later in this paper, Wiener's ideas continue to play an important role in understanding self-regulation and movement.
Following up on this heritage, I bring in findings from recent research in kinesiology, neurophysiology, and physical medicine to propose an understanding of the role of feedback in movement. The conference attendees included a wide-range of people interested in cybernetics: biologists, linguists, mathematicians, psychologists, computer scientists, visual and performing artists, systems scientists, family therapists, business consultants, and graduate students from various fields.
Building on the original insights of cybernetics and presenting research findings from three different areas of investigation-posture, balance, and disease, I offered a report from the field, expanding on cybernetic themes to offer insight into complexity of human movement and the nature of learning.
* * *
I am honored to be here today. Let me begin by thanking the organizers of this conference for giving me the opportunity to address you. In particular, I would like to acknowledge Robert Schoenholtz for organizing this plenary session.
I am interested in how people learn to move. Often, we think of learning movement as imitating what someone else does. However, research in the movement sciences has shown that demonstration and imitation are, at best, an unreliable strategy. Imitation may be the highest form of flattery, but it is the poorest form of teaching.
In my practice as a Feldenkrais teacher, I work with people who face different kinds of limitations in their movement abilities, ranging from performance problems in the arts or in sports to chronic pain and neurological disease.
The Feldenkrais Method of movement education consists of a systemic understanding of the human body's design for motion and a perception-oriented pedagogy for changing how people move (Feldenkrais, 1972; Goldfarb, 1990). Despite the best intentions, my students cannot move in the ways they want. The question that interests me, the question that I will address today, is how do people learn a movement if they cannot copy what they see?
The incredible complexity of movement is one of the first aspects that appears when one begins to consider movement, whether as an interested observer or as someone trying to deal with a problem or limitation. For instance, I raise my hand to adjust my glasses; in order to guide this movement, I must govern the action of my wrist, forearm, elbow, and shoulder.
Each of these joints allows different kinds and amounts of movements: since my wrist has two potential axes of motion, side-to-side and forward-and-back, kinesiologists say that it has two degrees of freedom. My forearm can only rotate around itself, thus it has one degree of freedom, like my elbow, which can only bend and straighten. Finally, my shoulder allows movement in three dimensions, thus it has three degrees of freedom.
To make sure that my hand reaches my glasses, I must regulate the amount of turning and/or bending in each of my arm's articulations, not to mention those of my fingers. Of course, I am not simply pushing my glasses up the bridge of my nose, but I maintain my balance and continue to talk to you while doing so. More on that in a few moments . . .
The only way I have of coordinating the movement of my arm is to regulate the relative contraction of my muscles. There are twenty-six muscles—ten at the shoulder, six at the elbow, four at the forearm, and six at the wrist—that govern the movement of the four joints we are considering. Like a string made up of braided threads, each of these muscles consists of hundreds of fibers.
Muscles are organized in bundles made up of groups of fibers and the nerve that activates those fibers. These functional groupings are known as motor units; the contraction of one of these bundles is smallest unit of independently regulated muscle action. The potential for independent action of each of these muscle bundles adds even more degrees of freedom.
You can see that this seemingly simple movement of my arm requires the complex temporal and spatial orchestration of all these muscle fibers: each joint must move the correct amount and at the right time in order for me to reach my destination and accomplish my task. In the movement sciences, the problem of managing this complexity is known as the degrees of freedom problem (Bernstein, 1967; Schmidt, 1988).
Degrees of Freedom
This is a familiar problem to any cybernetician: we recognize it as the problem that every complex system must encounter, the problem of governing its variety, the number of possible states that it can have (Beer, 1974). The human body, considered as a movement system, has an astronomical number of states with the hundreds of thousands of muscle fibers that make up the six hundred and ten muscles that regulate the motion at the joints formed by our two hundred and ten bones.
The manifold possibilities inherent in the incredible set of combinations afford a rich complexity to human movement. Such potential freedom of movement and expression poses a problem: how can we handle all this variety, all these degrees of freedom? To get a taste for this problem imagine for a moment trying to drive a car that, instead of having one steering wheel, had four steering devices, one for each wheel, and that each device controls the direction of each wheel independently.
Rather than one degree of freedom, this kooky car would have four degrees of freedom. Imagine, if you will, trying to drive this car around a corner or to parallel part. How could you do it?
This imaginary vehicle points out the difficulty of driving—or regulating—a multi-degree of freedom system like the human body, which is the problem that the nervous system constantly confronts. When we consider the complicated nature of everyday movements—standing and talking and pushing my glasses up my nose—there are simply too many pieces for all of them to be regulated independently, it is computationally impossible.
The potential degrees of freedom must, in any particular action, be temporarily reduced. This reduction of variety is accomplished by adding constraints. In a car, the complexity of steering is simplified: the two front wheels are yoked together so that they cannot act independently and the combined action of these wheels regulated by the one steering wheel. The back wheels can only spin, and they do not play an active role in the steering of the car. What mechanism could exist for decreasing the number of degrees of freedom involved in movement? Can we imagine an analogous, but momentary, constraint that would serve as a mechanism for simplifying the complexity of human movement?
Equilibreum
Researcher Lewis Nashner and his colleagues conducted investigations in the strategies that people use for the maintaining standing equilibrium (Nashner and McCollum, 1985; Horak and Nashner, 1986). In these experiments, Nashner studied how people maintained their balance while standing on a platform that was moved suddenly. In order to study the responses, the experimenters attached small electrodes on each subject's skin. These recording electrodes were placed over the muscles of the abdomen, back, buttocks, thighs, and lower legs. Since the electrical activity of muscles changes as their performance changes, the electrodes would record the activity of the muscles. Through this kind of procedure, known as electromyography, the experimenters could look at how the muscles functioned in response to a perturbation in balance.
The electromyographic record of different muscles' activation was correlated with recordings of the subjects' weight shifts (made by a force platform that records the distribution of pressure under the feet), and with the kinetic analysis of concurrent video records. Would the subjects show a random set of muscular contractions, different perhaps during each trial and different from person to person? Or would they exhibit some kind of distinct, stereotypical activation pattern?
Nashner's experiments demonstrated that, out of all the possible responses, compensation for the small disturbances in the moving-platform experiments involved movements of the hip and ankle. Indeed, there were two distinct compensatory patterns of muscular activation, which he referred to as hip and ankle strategies.
All compensatory responses involved either pure hip movement, pure ankle movement, or some combination of the two. The muscles of the trunk, hip, knee, and ankle acted together to control the joints they spanned and their related body segments in what are called coordinative structures or muscle synergies.
A muscle synergy constrains the relative activity of muscle groups that span two or more joints, thereby linking them together. The existence of synergetic patterns offers a solution for the degrees of freedom problem by simplifying the problem of movement coordination. Rather than deciding what every muscle should be doing at any one time, the nervous system only needs to make a decision about which coordinating configuration should be used. Evidence of coordinative structures has been demonstrated in other activities, such as walking (Shapiro, Zernicke, Gregor, & Diestel, 1981) and marksmanship (Tuller, Turvey, & Fitch, 1982).
You could well ask how Nashner's experiment relates to what happens in everyday life. Though we may rarely stand on surfaces that move unexpectedly, we are constantly maintaining our equilibrium. We are never still. Due to our body's physiological actions, including pumping blood, circulating cerebral spinal fluid, and breathing as well as the constant oscillation of muscle tissue (known as muscle tremor), we move continually.
postural sway and orientation
All of these movements are relevant because the human body is not designed to be static and still. We have three relatively large masses-the head, the torso, and the legs-each balanced on relatively small bases of support: 1) the skull on the spine, 2) the torso (and head) on the legs, and 3) the legs (and upper body) on the feet. Consequently even the slightest movement can throw us off-balance. While this is inferior design for static stability, it is an ideal design for movement: we have a high potential energy and therefore it takes only a minimal effort to start moving. The constant, unpredictable physiological motions of life, not to mention any small or large intentional actions, require us to actively regain our balance in order to maintain a standing posture.
Standing, therefore, is not static. Standing is a dynamic, bounded activity, one that can be described as keeping the center of gravity within stability boundaries, the crossing of which would make one fall over. What we have is a dynamic system that is never still, never quiet. We can see this constant act of balancing in the incessant small amplitude, high frequency movement known as postural sway. Movement scientists have noticed postural sway for quite a long time. It shows up in experiments that record the movement of the center of pressure over the feet and in kinematic analyses that record the small movements of the body over time.
This small wobble has, generally, been regarded as noise generated by an imperfect system and ignored. However, postural sway does not vary haphazardly, but rather, it varies within specific limits around a balance point. That is to say, this continual wobbling demonstrates that global minima and a performance envelope are persistent properties that are independent of quantitative details.
These persistent properties are the kind of topological invariance studied by chaos theory and qualitative dynamics. What's more, these studies have shown that the attractor demonstrates functional sensitivity, changing according to the activity in which the mover engages (Riccio, Lee, & Martin, in press).
Given the precarious nature of our standing balance, the regulation of movement cannot simply be a set of commands that determines the direction and extent of movement. In order to maintain equilibrium, the regulation of movement must also take into account the mechanical consequences of movements, the inertial forces generated, and the resulting changes in position. To what aspects of sensory experience do we orient ourselves in order to maintain equilibrium?
The question of orientation in our perception of standing or falling has historically been dismissed in scientific investigations. The classical argument has been that we locate ourselves within some objective or external frame of reference, such as gravity. If this were so, then perhaps postural sway could be ignored as meaningless noise yet the question of how we correct for constant planned and unplanned disturbances remains unanswered. What kind of feedback do we need?
In normal standing, the more off-balance we go, the easier it is gets to go further off-balance. If the total body mass is not aligned exactly over the base of support at the feet, gravitational forces act to further increase the misalignment and we accelerate away from equilibrium. Here we have a situation in which movement in different directions has different consequences: movement away from the balance point is easy; movement back towards equilibrium requires effort.
That movement is not as easy in every direction creates an asymmetrical topology, a qualitative direction of balance that we notice because our vestibular system is sensitive to changing acceleration and because our muscles can sense the increased effort required to stay vertical. The question is, are we sensitive to this?
In order to test if these qualitative dynamic characteristics play a role in balance, Riccio and his colleagues (Riccio, Martin, & Stoffregen, 1992) devised an experiment using a roll-axis tracking simulator (RATS). Used in flight simulation, the RATS apparatus consists of a seat in a carriage that can be tilted side-to-side. Each subject was securely strapped into a seat and blindfolded to eliminate the visual contribution to balance. The subjects held a joystick that allowed them to control the roll of the device and were instructed to keep themselves upright while the carriage was subjected to perturbations. The subjects did not know that the experimenters had manipulated the balance point and the dynamics of the device so that the direction of balance was skewed, that is, the direction of balance was no longer parallel with the direction of gravity.
Thanks to the strong motor and the computer controlling it, the RATS had an artificial balance point that replicated the characteristics of a natural balance point. Therefore, the further the device was moved from its artificial balance point, the more off-balance it behaved.
Sitting in the carriage, subjects were exposed to constant disturbances, which, if uncorrected, would lead to the box falling over. Could the subject's control the balance of the RATS with this artificial balance point? Yes. Even more interestingly, did the subjects perceive themselves as upright or not? When subjects were asked to estimate their tilt with respect to upright, their reports demonstrated that balance predominated over gravity in the influence of uprightness. That is, the tilted, artificial balance felt upright.
What does this mean? If the subjects had oriented to gravity, they would have fallen over, but keeping the device balanced required that they orient to something else. The subjects must have perceived upright as the result of their vestibularly based perception of the balance dynamics of the experimental device, the only "something else" to which they had access. Accordingly, the experience of being upright is not based on some objective, external frame of reference; rather, it is a consequence of ongoing perceptual-motor activity. The asymmetrical dynamics of balance define a state space that the subjects were sensitive to and dependent upon. In other words, the perception of upright is dependent on perceiving qualitative changes in dynamics.
Perception is ongoing
This conclusion is consistent with second-order cybernetics where perception is understood as active rather than passive (von Foerster, 1984) and constructed perceptual experience is a result of the subject's activity. To investigate this idea, Riccio repeated the experiment, placing a subject in the RATS and having the computer play back the movements experienced by an active subject. In this condition, subjects went through the same exact motions as subjects who actively maintained the device's balance, but they had no control over the RATS. Did the "passengers" have the same experience as the subjects who had control their motion? As it turns out, the passive observers were less certain of upright and were uninfluenced by the direction of balance.
Wiener suggested this kind of active perception in Cybernetics (1948):
"Another interesting variant of feedback systems is found in the way in which we steer a car on an icy road. Our entire conduct of driving depends on a knowledge of the slipperiness of the road surface, that is, on a knowledge of the performance characteristics of the system car-road. If we wait to find this out by the ordinary
performance of the system, we shall discover ourselves in a skid before we know it. We thus give to the steering wheel a succession of small, fast impulses, not enough to throw the car into a major skid but quite enough to report to our kinesthetic sense whether the car is in danger of skidding, and we regulate our method of steering accordingly." (p. 113)
Wiener referred to this kind of feedback as informative feedback. In the movement sciences, we draw a distinction between performative and exploratory movements (Riccio, in press). Performative movements are those in which the mover accomplishes some goal or aim, such as driving a car. The feedback necessary for guiding performative movements provides a report on the progress toward the goal. Exploratory movements inform the mover about moment-to-moment consequences of an activity and orient the mover to a relationship with the environment, such as testing the iciness of the road by gently oscillating the wheel. Thus, exploratory movements provide informative feedback.
Variability and control problems
Exploratory movements and informative feedback are important for systems that face changing and unpredictable situations. Only by introducing slight perturbations into constantly changing circumstances can a dynamic system keep track of its present state. The danger is that, if they were too large, these exploratory perturbations could interfere with whatever activities the system is performing; in control engineering, this problem is known as the dual control problem. Returning to the example of the icy road, this means that if the oscillations on the steering wheel are too big, they will interfere with the driver's ability to control the car. Therefore, exploratory perturbations must be of a different magnitude than the performative movements so as not to interfere with their realization.
Now if we return to postural sway, we might suppose that this variability in movement is not noise. Rather, perhaps the incessant wobble of human beings is informative. The nervous system must have some way of knowing the current status of the body in respect to equilibrium on a moment-to-moment basis and, as I have said, the body is neither static nor predictable. We know from Riccio's experiment that: "ubiquitous movements can provide dynamical information without interfering with controlled adjustments in posture (Riccio, in press, p. 29). " Therefore, humans are sensitive to dynamics that provide the information necessary for maintaining balance.
What a marvelous adaptation to use the system's constant pulses of-provided by physiological processes—to generate the informative feedback needed to monitor the system's current and ever—changing state. Postural sway functions as kind of exploratory movement that allows the system to learn about where it stands.
This constant exploring of the present configuration is what gives us the ability to simultaneously stand and act. Constructing a system that controls for the consequences of ongoing activity is a difficulty in designing robots. The problem lies in the nature of a multifunction automaton that needs to simultaneously maintain balance and engage in other activities. It is a much simpler control problem to fabricate a machine that has a stable base and performs sequential tasks.
However, a robot built on the physical blueprint of the human body-one that would engage in multiple, overlapping, and possibly contradictory activities-has not been achieved because it requires an adaptive control system that receives constant informative feedback. This is an incredibly complex task, which is one reason why present-day specimens resemble the robot from the old "Lost In Space" series, with its large base of support and rigid body, rather than the android Data, with his human frame and mutli-link body, from the current TV show, Star Trek, The Next Generation.
To recap, I have suggested that posture and movement are steered by synergetic patterns of muscle activity and that the guidance of movement is dependent on sensing dynamic qualities. There are situations when this system seems to not function properly, when people's ability to move and control their movements is limited. For instance, I have worked extensively, both in clinical settings and in private practice, with individuals who have chronic back pain. In the most common situation, these people suffer from prolonged muscle spasms. Such local hypertonic states are often considered both protective and problematic (Janda, 1992): immediately following an injury, the person unconsciously splints, or braces, in order to deal with pain elicited by moving.
This stiffening can be seen as a limiting, or freezing, of degrees of freedom, to make the control of movement simple under duress. The splinting leads to impaired movement and impaired movement makes the person more constrained, less able to adapt to his or her environment and more prone to re-injury. Even worse, the stiffness leads to greater discomfort that leads to further contraction that leads to recurrent pain. This vicious circle—what we call a positive feedback loop (Maruyama, 1968)—is evident in many difficult cases of chronic pain.
Vladimir Janda and his colleagues have been demonstrated the existence of persistent forms of muscle spasm (Janda, 1980). In one experiment, Janda compared normal subjects with those who had chronic back pain and spasm.
Both groups performed abdominal strengthening exercises—modified sit-ups commonly prescribed to patients with back pain—while the muscular activity of the stomach and back was monitored through electromyography. As expected, the normal subjects exhibited pattern of muscle contraction known as reciprocal inhibition: when they contracted their abdominal muscles, their back muscles relaxed. The back patients demonstrated a co-contraction pattern: while contracting their abdominal muscles, they contracted their back muscles even more strongly. In this situation, well-meaning attempts to strengthen abdominal muscles led to the already tight back muscles getting all that much tighter.
Splinting and co-contraction
Unlike the situation in which someone does many activities at the same time so that the activities work together, splinting illustrates what happens when someone has two simultaneous and conflicting goals.
It is easy enough to surmise that the habitual pattern of muscle tightness associated with splinting interferes not only with performance, but also with exploration. Constant muscle tightness impedes postural sway and hinders exploration, biasing kinesthesia by altering the dynamics of exploration. This bias affects the operation of the sensory-motor loop, and disturbs the perception of posture and motion. Furthermore, the prolonged nature of such muscle stiffness will lead to habituation and habituation leads to what we could call a dynamical blindspot (von Foerster, 1984), where the mover is unaware of both the tightness and the perceptual distortions it causes, as well as being unaware that he or she is unaware.
The importance of exploratory movements in perception and coordination is generally not appreciated. We are unaware of the ongoing wobble of standing and the role it normally plays in our marvelous sensitivity to balance. This situation is similar to findings about tactile perception, where movement is required for sensitivity but where the role of that movement plays in generating sensation is not understood. When we touch a surface, we are left with an impression of its texture, an impression that does not include a memory of the movements involved in touching (Katz, 1989).
In Wiener's driving example, we would describe the road as slippery without necessarily being aware of the movement of our hands on the steering wheel, movements that led to that perception. The same thing is true in our kinesthetic perception: we notice an external world, but we do not appreciate the active exploratory process that makes noticing possible.
Not appreciating the importance of feedback and exploratory movement in coordination leads to difficulties in understanding and solving movement problems. Often the person with back pain is instructed to move differently to avoid pain and future injury. It is common to prescribe exercise to stretch tight muscles and strengthen weak ones, a therapeutic method that reflects the mind- body dualism inherent in orthodox medical thought. Tight muscles are not the cause of the problem; rather they are a consequence of the ongoing functioning of the nervous system.
The underlying problem is a perceptual problem. The person with back pain cannot accurately perceive his or her own movement, and without reliable perceptions, simply cannot govern that movement. The needed learning lies in improving perception, particularly kinesthesia, and improving perception means engaging in exploratory activity to break the sensory blindness due to habituation. Immobilization with a corset or traction, another common back treatment, is precisely wrong since it limits exploratory movement.
learning is understanding what to change
The problem encountered in learning a new way of moving is that an old and often habitual pattern interferes. It is not possible to simply do something new.
Learning a new movement means learning about what is already happening, so as to understand what to change. It also means figuring out which aspects of dynamics to attend to, that is, learning a new movement means learning what to notice. If, as we say, information is a difference that makes a difference (Bateson, 1979), then learning is finding out which differences matter. This is where my work with the Feldenkrais Method of movement education comes into play, for Feldenkrais uses gentle movements to explore the quality of movement and to refine kinesthetic perception.
The method is based on the insight that where there are frozen degrees of freedom-where movement is more limited than the skeleton requires it to be—there are perceptual distortions. This link between undifferentiated movement and undifferentiated sensation leads to teaching movement by refining perception; perception, in turn, is refined through informative feedback by intentionally engaging in exploratory movement.
As a Feldenkrais teacher, I orient my students, either through direct manual contact or through verbal instructions, to explore specific aspects of movement. These explorations are designed to develop the students' awareness of habitual muscle synergies and create patterns of movement. For someone with a back problem, the process of learning focuses on becoming aware of the current pattern of muscle stiffness and its inhibitory consequences.
One resensitizing strategy is to have the student perform unusual movements in unlikely positions. This serves to break habituation and introduce new ways of perceiving. Another strategy is to have the student explore while in an intentionally constrained position; the addition of constraints both simplifies the motor control task and highlights specific aspects of dynamics. Such constraints inhibit habitual action and free up unused degrees of freedoms. Emphasis is placed on exploration, not performance. The student is asked to move slowly and gently so that he or she can tune into the qualitative aspects of the movement.
Can these ideas suggest strategies for learning in general? In my experience, most people equate learning with practice. This is a Catch 22: you can't practice something that you don't know how to do. This is especially true for someone who has a movement limitation and cannot produce the desired movement. I am suggesting that learning is not practicing and learning is also not performing. Learning is exploring. A teacher's job is to create experiences that guide the learner's exploration.
These investigations—based on the positions described and the inquiries suggested—structure the domain of exploration and provide students with clues about what is important to notice. This approach is applicable in any domain of learning.
Reference List
Adams, J. A. (1971) “A closed-loop theory of motor learning.” Journal of Motor Behavior, 3 (2), 111-149.
Bateson, Gregory (1979). Mind and nature. New York: Dutton.
Beer, Stafford. (1974) Designing freedom. Toronto: CBC Publications.
Bernstein, N. (1967) The coordination and regulation of movements. Oxford: Pergamon Press.
Feldenkrais, M. (1972) Awareness through movement. New York: Harper & Row. Goldfarb, L. W. (1990) Articulating changes. Berkeley: Feldenkrais Resources.
Horak, F. B. and Nashner, L. M. (1986) “Central programming of postural movements: adaptations to altered support-surface configurations.” Journal of Neurophysiology, 55 (6), 1369-1381.
Janda, V. (1992) “Treatment of chronic back pain.” Journal of Manual Medicine, 6 166-168.
Janda, V. (1980) “Muscles as a pathogenic factor in back pain.” In The treatment of patients (pp. 2-23). Christchurch, New Zealand: International Federation of Orthopaedic Manipulative Therapists.
Katz, D. (1989) The world of touch (L. E. Krueger Trans.). Hillsdale, NJ: Lawrence Erlbaum.
Maruyama, M. (1968) “The second cybernetics: deviation amplifying mutual causal processes.” In W. Buckley (Ed.) Modern systems research for the behavioral scientist (pp. 304-313). Chicago: Aldine.
Nashner, L. M. and McCollum, G. (1985) “The organization of human postural movements: A formal basis and experimental synthesis.” The Behavioral and Brain Sciences, 8, 135-172
Riccio, G. E. (in press) “Coordination of postural control and vehicular control: implications for multimodal perception and simulation of self motion. In J. Flach, P. Hancock, J. Caird, & K. Vicente (Eds.), The ecology of human-machine systems. Hillsdale, NJ: Lawrence Erlbaum.
Riccio, G. E., Martin, E. J., & Stoffregen, T. E. (1992) “”The role of balance dynamics in the active perception of orientation.” Journal of Experimental Psychology: Human Perception and Performance, 18 (3), 624-644.
Schmidt, R. A. (1988) Motor control and learning. 2nd edition. Champaign, IL: Human Kinetics Publishers.
Shapiro, D. C., Zernicke, R. F., Gregor, R. J., & Diestel, J. D. (1981). “Evidence for generalized motor programs using gait-pattern analysis.” Journal of Motor Behavior, 13, 33-47.
Turvey, M.T., Saltzman, E, and Schimdt, R.C. (1991) “Dynamics and task-specific coordinations.” In N. Badler, B. A. Barsky, & D. Zeltzer (Eds.) Making them move (pp. 157-170). San Mateo: Morgan Kaufmann Publishers.
von Foerster, H. (1984) “On constructing a reality.” In Observing systems (2nd ed.) (pp. 233-309). Seaside, CA: Intersystems Press.
Wiener, Norbert. (1961) Cybernetics (2nd ed.). Cambridge: MIT Press.