The Hierarchical Behavior of Perception

A slightly expanded version of this paper was published in Closed Loop,Journal of Living Control Systems, Vol 3 No 4, Fall 1993. For back issuesof this journal, see INTROCSG.NET.

For more experiments and Fig 2.,

see Levels of Intention in Behavior

in Mind Readings: Experimental Studies of Purpose.

Details in file MIND_RD.INF.

THE HIERARCHICAL BEHAVIOR OF PERCEPTION

By Richard S. Marken 1991

Abstract

This paper argues that the coincidental development of hierarchical modelsof perception and behavior is no coincidence. Perception and behavior are twosides of the same phenomenon --control. A hierarchical control system model shows that evidence ofhierarchical organization in behavior is also evidence of hierarchicalorganization in perception. A number of studies of the temporal structure ofbehavior are shown to be consistent with studies of the temporal structure ofperception. A surprising implication of the control model is that temporallimitations of behavior are based on temporal limitations of perception.Action systems cannot produce controlled behavioral results faster than therate at which these results can be perceived. Behavioral skill turns on theability to control a hierarchy of perceptions, not actions.

Introduction

Psychologists have developed hierarchical models of both perception (eg.Bryan and Harter, 1899; Palmer, 1977; Simon, 1972; Povel, 1981) and behavior(eg. Albus, 1981; Arbib, 1972; Greeno and Simon, 1974; Lashley, 1951; Martin,1972; Keele, Cohen and Ivry, 1990; Rosenbaum, 1987). This could be acoincidence, a case of similar models being applied to two very differentphenomena. On the other hand, it could reflect the existence of a common basisfor both perception and behavior. This paper argues for the latterpossibility, suggesting that perception and behavior are two sides of the samephenomenon --control (Marken, 1988). Control is the means by which organisms keep perceivedaspects of their external environment in desired states (Powers, 1973). Theexistence of hierarchical models of both perception and behavior is a result oflooking at control from two different perspectives; that of the person doingthe controlling (the actor) and that of the person watching control (theobserver). Depending on the perspective, control can be seen as a perceptualor a behavioral phenomenon.

From the actor's perspective, control is a perceptual phenomenon. Theactor is controlling his or her own perceptual experience, making it behave asdesired. However, from the observer's perspective, control is a behavioralphenomenon. The actor appears to be controlling variable aspects of his or herbehavior in relation to the environment. For example, from the perspective ofa typist (the actor), typing involves the control of a dynamically changing setof kinesthetic, auditory and, perhaps, visual perceptions. If there were noperceptions there would be no typing. However, from the perspective of someonewatching the typist (the observer), perception is irrelevant; the typistappears to be controlling the movements of his or her fingers in relation tothe keys on a keyboard.

These two views of control have one thing in common; in both cases, controlis seen in the behavior of perception. For the actor, control is seen in thebehavior of his or her own perceptions. For the observer, control is seen inthe behavior of his or her own perceptions of the actor's actions. (Theobserver can see the means of control but can only infer their perceptualconsequences as experienced by the actor). If control is hierarchical then itcan be described as the behavior of a hierarchy of perceptions. Hierarchicalmodels of perception and behavior can then be seen as attempts to describecontrol from two different perspectives, those of the actor and observer,respectively. This paper presents evidence that hierarchical models ofperception and behavior reflect the hierarchical structure of control.

A Perceptual Control Hierarchy

The concept of control as the behavior of perception can be understood inthe context of a hierarchical control system model of behavioral organization(Powers, 1973; 1989). The model is shown in Figure 1. It consists of severallevels of control systems which control perceptions of different aspects of theexternal environment. All systems control perceptions in the same way; byproducing actions that reduce the discrepancy between actual and intendedperceptions. Intended perceptions are specified by the reference inputs to thecontrol systems. The actions of the control systems coax perceptual inputsinto a match with reference inputs via direct or indirect effects on theexternal environment. The actions of the lowest level control systems affectperceptions directly through the environment. The actions of higher levelcontrol systems affect perceptions indirectly by adjusting the reference inputsto lower level systems.

-------------------------------

Insert Figure 1 Here

See Living Control Systems I, page 278

-------------------------------

The hierarchy of control systems is a working model of purposeful behavior(Marken, 1986; 1990). The behavior of the hierarchy is purposeful inasmuch aseach control system in the hierarchy works against any opposing forces in orderto produce intended results. Opposing forces come from disturbances created bythe environment as well as interfering effects caused by the actions of othercontrol systems. The existence of disturbances means that a control systemcannot reliably produce an intended result by selecting a particular action.Actions must vary to compensate for varying disturbances. Control systemssolve this problem by specifying what results are to be perceived, not howthese results are to be achieved. Control systems control perceptions, notactions. When set up correctly the control systems in the hierarchy vary theiractions as necessary, compensating for unpredictable (and, often, undetectable)disturbances, in order to produce intended perceptions. Indeed, the term"control" refers to this process of producing intended perceptions in adisturbance prone environment.

Levels of Perception.

Powers (1990) has proposed that each level of the hierarchy of controlsystems controls a different class of perception. These classes representprogressively more abstract aspects of the external environment. The lowestlevel systems control perceptions that represent the intensity of environmentalinput. The next level controls sensations (such as a colors), which arefunctions of several different intensities. Going up from sensations there iscontrol of configurations (combinations of sensations), transitions (temporalchanges in configurations), events (sequences of changing configurations),relationships (logical, statistical, or causal co-variationbetween independent events), categories (class membership), sequences (uniqueorderings of lower order perceptions), programs (if-thencontingencies between lower level perceptions), principles (a general rule thatexists in the behavior of lower level perceptions) and system concepts (aparticular set of principles exemplified by the states of many lower levelperceptions; see Powers, 1989, pp. 190-208). These eleven classes of perception correspond to eleven levels of controlsystems in the hierarchical control model. All control systems at a particularlevel of the hierarchy control the same class of perception, though each systemcontrols a slightly different exemplar of the class. Thus, all systems at aparticular level may control configuration perceptions but each system controlsa different configuration.

The rationale for hierarchical classes of perceptual control is based onthe observation that certain types of perception depend on the existence ofothers. Higher level perceptions depend on (and, thus, are a function of)lower level perceptions. For example, the perception of a configuration, suchas a face, depends on the existence of sensation (color) or intensity(black/white) perceptions. The face is a function of these sensations andintensities. The lower level perceptions are the independent variables in thefunction that computes the higher level perception. Their status asindependent variables is confirmed by the fact that lower level perceptions canexist in the absence of the higher level perceptions, but not vice versa.Color and intensity perceptions can exist without the perception of a face (orany other configuration, for that matter) but there is no face withoutperceptions of intensity and/or color.

The Behavior of Perceptions.

From the point of view of the hierarchical control model, "behaving" is aprocess of controlling perceptual experience. Any reasonably complex behaviorinvolves the control of several levels of perception simultaneously. Forexample, when typing the word "hello", one controlled perception is thesequence of letters "h", "e", "l" ,"l" and "o". The perception of thissequence is controlled by producing a sequence of keypress event perceptions.Each keypress event is controlled by producing a particular set of transitionsbetween finger configuration perceptions. Each finger configuration iscontrolled by a different set of force sensations which are themselvescontrolled by producing different combinations of intensities of tensions in aset of muscles.

The perceptions involved in typing "hello" are all being controlledsimultaneously. Transitions between finger configurations are being controlledwhile the force sensations that produce the configuration perceptions are beingcontrolled. The typist is not necessarily aware of the behavior of all theselevels of perception. When people type they are probably only aware of thehighest level perceptions that they intend to produce, such as the word theyintend to type. Nevertheless, people can direct their attention to thedifferent levels of perception involved in behavior. For example, it ispossible to attend to perceptions of muscle tension, finger movement and fingertip pressure that are produced while typing.

People do not ordinarily attend to the behavior of their perceptionsbecause doing so leads to a deterioration of performance. Paying attention toone's own behavior in this way is the opposite of "zen" behavior, where youjust attend to the particular (perceptual) results that you intend to produce,letting the lower level perceptions required to produce these results occurautomatically (Herrigal, 1971). While it violates the principles of zen,attention to the perceptions involved in the production of behavioral resultscan provide interesting hints about the nature of the perceptual controlhierarchy.

The Perception of Behavior.

The behavior of an actor who is organized like the hierarchical controlmodel consists of changes in the values of variables in the actor'senvironment. An observer cannot see what is going on inside the actor; he orshe can only see the actor's actions and the effect of these actions on theexternal environment. The effect of these actions is to cause purposefulbehavior of certain variables in the environment; the variables that correspondto perceptions that the actor is actually controlling. The purposefulness ofthe behavior of these variables is evidenced by the fact that consistentbehaviors are produced in the context of randomly changing environmentaldisturbances. Thus, a typist can consistently type the word "hello" despitechanges in the position of the fingers relative to the keyboard, variations inthe push-backforce of the keys or even a shift from one keyboard arrangement to another(from QWERTY to Dvorak, for example).

Since the actor controls his or her own perceptions, the observer cannotactually see what the actor is "doing"; the actor's "doings" consist ofchanging the intended states of his or her own perceptions. All the observersees is variable results of the actor's actions; results that may or may not beunder control. For example, the observer, might notice that a click occurseach time the typist presses a key. The click is a result produced by thetypist and the observer is likely to conclude that the typist is controllingthe occurrence of the click. In fact, the click may be nothing more than aside effect of the typist's efforts to make the key feel like it has hitbottom. There are methods that make it possible for the observer to tellwhether or not his or her perceptions of the actor's behavior correspond to theperceptions that are being controlled by the actor (Marken, 1989). Thesemethods make it possible for the observer to determine what the actor isactually doing (i.e. controlling).

Hierarchical Control

The hierarchical nature of the processes that generate behavior would notbe obvious to the observer of a hierarchical control system. The observercould tell that the system is controlling many variables simultaneously but heor she would find it difficult to demonstrate that some of these variables arebeing controlled in order to control others. For example, the observer couldtell that a typist is controlling letter sequences, keypress events, fingermovements and finger configurations. But the observer would have a hard timeshowing that these variables are hierarchically related. The observer couldmake up a plausible hierarchical description of these behaviors; for example,finger positions seem to be used to produce finger movements which are used toproduce keypresses which are used to produce letter sequences. But finding ahierarchical description of behavior does not prove that the behavior isactually produced by a hierarchical process (Davis, 1976; Kline, 1983).

Hierarchical Invariance

Hierarchical production of behavior implies that the commands required toproduce a lower level behavior are nested within the commands required toproduce a higher level behavior. For example, the commands that produce aparticular finger configuration would be nested within the commands thatproduce a movement from one configuration to another. Sternberg, Knoll andTurlock (1990) refer to this nesting as an invariance property of hierarchicalcontrol. Lower level commands are like a subprogram that is invoked by aprogram of higher level commands. The invariance of hierarchical controlrefers to the assumption that the course of such a subprogram does not dependon how it was invoked from the program (low level invariance); similarly, thecourse of the program does not depend on the nature of the commands carried outby the subprograms (high level invariance).

Convergent and Divergent Control.

The hierarchical control model satisfies both the low and high levelinvariance properties of hierarchical control. The commands issued by higherlevel systems have no effect on the commands issued by lower level systems andvice versa. It is important to remember, however, that the commands in thecontrol hierarchy are requests for input, not output. Higher level systemstell lower level systems what to perceive, not what to do. This aspect ofcontrol system operation solves a problem that is either ignored or glossedover in most hierarchical models of behavior: How does a high level command getturned into the lower level commands that produce results that satisfy the highlevel command? If commands specify outputs then the result of the same commandis always different due to varying environmental disturbances. The high levelcommand to press a key, for example, cannot know which lower level outputs willproduce this result on different occasions. This problem is solved by thehierarchical control model because intended results are represented as aconvergent function rather than a divergent network.

Most hierarchical models of behavior require that a high level command bedecomposed into the many lower level commands that produce the intended result.In the hierarchical control model, both the high level command and the intendedresult of the command are represented by a single, unidimensional signal. Thesignal that represents the intended result is a function of results produced bymany lower level commands. But the high level command does not need to bedecomposed into all the appropriate lower level commands (Powers, 1979). Thedifference between the high level command and the perceptual result of thatcommand is sufficient to produce the lower level commands that keep theperceptual result at the commanded value (Marken, 1990).

Levels of Behavior

The hierarchical invariance properties of the control hierarchy provide abasis for determining whether its behavior is actually generated byhierarchical processes. Hierarchical control can be seen in the relativetiming of control actions. In a control hierarchy, lower level systems mustoperate faster than higher level systems. Higher level systems cannot producea complex perceptual result before the lower level systems have produced thecomponent perceptions on which it depends. This nesting of control actions canbe seen in the differential speed of operation of control systems at differentlevels of the control hierarchy. Lower level systems not only correct fordisturbances faster than higher level ones; they carry out this correctionprocess during the higher level correction process. The lower level controlprocess is temporally nested within the higher level control process.

Arm Movement.

Powers, Clark and McFarland (1960) describe a simple demonstration ofnested control based on relative timing of control system operation. A subjectholds one hand extended straight ahead while the experimenter maintains a lightdownward pressure on it. The subject is to move his or her arm downward asquickly as possible when the experimenter signals with a brief, downward pushon the subject's extended hand. The result of this simple experiment is alwaysthe same: the subject responds to the downward signal push with a brief upwardpush followed by downward movement of the arm. An electromyograph shows thatthe initial upward push is an active response and not the result of muscleelasticity.

The arm movement demonstration reveals one level of control nested withinanother. The subject's initial upward push (which cannot be suppressed) is thefast response of a lower level control system that is maintaining theperception of arm position in a particular reference state (extended forward).The behavior of this system is nested within the response time of a higherlevel system that moves the arm downward. The higher level system operates bychanging the reference for the arm position control system. The downwardsignal push causes the brief upward reaction because the signal is treated as adisturbance to arm position. This is particularly interesting because thesignal is pushing the arm in the direction it should move; the lower levelreaction is "counter productive" with respect to the goal of the higher levelsystem (which wants to perceive the arm down at the side). The reaction occursbecause the lower level system starts pushing against the disturbance to armposition before the higher level system can start changing the reference forthis position.

Polarity Reversal.

A more precise test of nested control were performed in a series ofexperiments conducted by Marken and Powers (1989). In one of theseexperiments, subjects performed a standard pursuit tracking task, using a mousecontroller to keep a cursor aligned with a moving target. At intervals duringthe experiment the polarity of the connection between mouse and cursor movementwas reversed in a way that did not disturb the cursor position. Mousemovements that had moved the cursor to the right now moved it to the left;mouse movements that had moved the cursor to the left now moved it to theright.

A sample of the behavior that occurs in the vicinity of a polarity reversalis shown in Figure 2. The upper traces show the behavior of a control systemmodel and the lower traces show the behavior of a human subject. When thereversal occurs, both the model and the subject respond to error (the deviationof the cursor from the target) in the wrong direction, making it larger insteadof smaller (any deviation of the error trace from the zero line represents anincrease in error). The larger error leads to faster mouse movement whichcauses the error to increase still more rapidly. A runaway condition ensueswith error increasing exponentially.

--------------------------------

Figure 2 Here.

See Levels of Intention in Behavior, Figure 2 on page 116

in Mind Readings: Experimental Studies of Purpose.

---------------------------------

About 1/2 second after the polarity reversal the subject's behavior departsabruptly from that of the model. The subject adjusts to the polarity reversaland the error returns to a small value. The model cannot alter itscharacteristics and the error trace quickly goes off the graph. These resultsprovide evidence of two nested levels of control operating at different speeds.The faster, lower level system control the distance between cursor and target.This system continues to operate as usual even when, due to the polarityreversal, this causes an increase in perceptual error. Normal operation isrestored only after a slower, higher level system has time to control therelationship between mouse and cursor movement.

Levels of Perception

The arm movement and polarity shift experiments reveal the hierarchicalorganization of control from the point of view of the observer. Thehierarchical control model suggests that it should also be possible to viewhierarchical organization from the point of view of the actor. From theactor's point of view, hierarchical control would be seen as a hierarchy ofchanging perceptions. One way to get a look at this hierarchy is again interms of relative timing; in this case, however, in terms of the relativetiming of the perceptual results of control actions rather of the actionsthemselves.

Computation Time Window.

The hierarchical control model represents the results of control actions asunidimensional perceptual signals. A configuration, such as the letter "h", isa possible result of control actions, as is a sequence of letters, such as theword "hello". The model represents these results as perceptual input signals,the intensity of a signal being proportional to the degree to which aparticular result is produced. This concept is consistent with thephysiological work of Hubel and Wiesel (1979) who found that the firing rate ofan afferent neuron is proportional to the degree to which particularenvironmental event occurs in the "receptive field" of the neuron.

Many of the higher level classes of perception in the control hierarchydepend on environmental events that vary over time. Examples are transitions,events, and sequences. The neural signals that represent these variables mustintegrate several lower level perceptual signals that occur at different times.Hubel and Weisel found evidence of a computation time window for integratingperceptual signals. Certain cells respond maximally to configurations (such as"lines") that move across a particular area of the retina at a particular rate.These are "motion detector" neurons. The neuron responds maximally to movementof a configuration that occurs within a particular time window. Movement thatoccurs outside of this time window is not included in the computation of theperceptual signal that represents motion.

Levels by Time

The hierarchical control model implies that the duration of the computationtime window increases as you go up the hierarchy. The computation time windowfor the perception of configurations should be shorter than the computationtime window for the perception of transitions which should be shorter than thecomputation time window for the perception of sequences. I have developed aversion of the psychophysical method of adjustment which makes it possible tosee at least four distinct levels of perception by varying the rate at whichitems occur on a computer display. A computer program presents a sequence ofnumbers at two different positions on the display. The presentation positionsare vertically adjacent and horizontally separated by 2 cm. The numbers arepresented alternately to the two positions. The subject can adjust the rate atwhich the numbers occur in each position by varying the position of a mousecontroller.

At the fastest rate of number presentation subjects report that the numbersappear to occur in two simultaneous streams. The fact that the numbers arepresented to the two positions alternately is completely undetectable.However, even at the fastest rate of number presentation subjects can make outthe individual numbers in each stream. At the fastest rate, there areapproximately 20 numbers per second in each stream. This means that there is a50 msec period available for detecting each number. This duration isapparently sufficient for number recognition suggesting that the computationtime window for perception of configuration is less than 50 msec. Studies ofthe "span of apprehension" for sets of letters suggest that the duration of thecomputation time window for perception of visual configuration may be even lessthan 50 msec, possibly as short as 15 msec (Sperling, 1960).

As the rate of number presentation slows the alternation between numbers inthe two positions becomes apparent. Subjects report perception of alternationor movement between numbers in the two positions when the numbers in eachstream are presented at the rate of about 7 per second. At this rate, analternation from a number in one stream to a number in another occurs in 160msec. This duration is sufficient for perception of the alternation as atransition or movement from one position to the other suggesting that thecomputation time window for transition perception is on the order of 200 msec.This duration is compatible with estimates of the time to experience optimalapparent motion when configurations are alternately presented in two differentpositions (Kolers, 1972).

The numbers presented in each stream are always changing. However,subjects find it impossible to perceive the order of the numbers as theyalternate from one position to another even though it is possible to clearlyperceive the individual numbers and the fact that they are alternating andchanging across positions. The rate of number presentation must be slowedconsiderably, so that each stream of numbers is presented at the rate of abouttwo per second, before it is possible to perceive the order in which thenumbers occur. At this rate numbers in the sequence occur at the rate of fourper second. These results suggest that the duration of computation time windowfor the perception of sequence is about 0.5 seconds. This is the time it takesfor two elements of the sequence to occur, the minimum number that canconstitute a sequence.

The numbers in the rate adjustment study did not occur in a fixed,repeating sequence. Rather, they were generated by a set of rules, a program.The sequence of numbers was unpredictable unless the subject could perceive therule underlying the sequence. One rule was: if the number on the right is eventhen the number on the left will be odd. The other rule was: if the number onthe left is greater than 5 then the one on the right will be less than 5.(Numbers in the sequence were also constrained to be between 0 and 9).Subjects could not perceive the program underlying the sequence of numbersuntil the speed of the two streams of numbers was about .5 numbers per secondso that the numbers in the program occurred once per second. The perception ofa program in a sequence of numbers requires considerably more time then ittakes to perceive the order of numbers in the same sequence.

The perception of a sequence or a program seems to involve more mentaleffort than the perception of a configuration or a transition. Higher levelperceptions, like programs, seem to represent subjective rather than objectiveaspects of external reality; they seem more like interpretations thanrepresentations. These higher level perceptions are typically called"cognitions". Of course, all perceptions represent subjective aspects ofwhatever is "out there"; from the point of view of the hierarchical controlmodel, the location of the line separating perceptual from cognitiverepresentations of reality is rather arbitrary. Behavior is the control ofperceptions which range from the simple (intensities) to the complex(programs).

General Sequence Perception Limits.

The hierarchical control model says that all perceptions of a particulartype are controlled by systems at the same level in the hierarchy. Thisimplies that the speed limit for a particular type of perception should beabout the same for all perceptions of that type. The 150 msec computation timewindow for perception of transition, for example, should apply to both visualand auditory transition. There is evidence that supports this proposition.Miller & Heise (1950) studied the ability to perceive an auditorytransition called a "trill". A trill is the perception of a temporalalternation from one sound sensation or configuration to another. The speedlimit for trill perception is nearly the same as the speed limit for visualtransition perception found in the number rate adjustment study --about 15 per second. As in the visual case, when the rate of alternation ofthe elements of the auditory trill exceeds the computation time window theelements "break" into two simultaneous streams of sound; the perception oftransition (trill) disappears even though the sounds continue toalternate.

There is also evidence that the four per second speed limit for sequenceperception found in the number rate adjustment study applies across sensorymodalities. Warren, Obusek, Farmer, & Warren (1969) studied subjects'ability to determine the order of the components sounds in a sound sequence.They found that subjects could not perceive the order of the components untilthe rate of presentation of the sequence was less than or equal to four persecond. This was a surprising result because it is well known that people candiscriminate sequences of sounds that occur at rates much faster than four persecond. In words, for example, the duration of the typical phoneme is 80 msecso people can discriminate sequences of phoneme sounds that occur at the rateof about 10 phonemes per second. But there is reason to believe that thephonemes in a word are not heard as a sequence; that is, the order of thephonemes cannot be perceived. Warren (1974) showed that subjects can learn totell the difference between sequences of unrelated sounds that occur at ratesof 10 per second. However, the subjects could not report the order of thesounds in each sequence; only that one sound event differed from another. Aword seems to be a lower order perception --an event perception --which is recognized on the basis of its overall sound pattern. There is noneed to perceive the order of the phonemes occur; just that the temporalpattern of phonemes (sound configurations) for one word differs from that forother words.

The Relationship Between Behavior and Perception

Configurations, transitions, events, sequences and programs are potentiallycontrollable perceptions. An actor can produce a desired sequence of sounds,for example, by speaking sound events (phonemes) in some order. An observerwill see the production of this sequence as a behavior of the actor. Thehierarchical control model suggests that the actor's ability to produce thisbehavior turns on his or her ability to perceive the intended result. Sinceperception depends on speed, it should be impossible for the actor to producean indented result faster than the result can be perceived. The observer willsee this speed limit as a behavioral limit. An example of this can be seen inthe arm movement experiment described above. In that experiment it appearsthat the time to respond to the signal push is a result of a behavioral speedlimit; the inability to generate an output faster than a certain rate. But acloser look indicates that the neuromuscular "output" system is perfectlycapable of responding to a signal push almost immediately, as evidenced by theimmediate upward response to the downward signal push. The same muscles thatproduce this immediate reaction must wait to produce the perception of the armmoving downward. The speed limit is not in the muscles. It is in the resultsthat the muscles are asked to produce; a static position of the arm (aconfiguration perception) and a movement of the arm in response to the signalpush (a relationship perception).

Sequence Production and Perception. Some of the most interesting thingspeople do involve the production of a sequence of behaviors. Some recentstudies of temporal aspects of sequence production are directly relevant to thehierarchical control model. In one study, Rosenbaum (1989) asked subjects tospeak the first letters of the alphabet as quickly as possible. When speed ofletter production exceeded four per second the number of errors (producingletters out of sequence) increased dramatically, indicating a loss of controlof the sequence. The speed limit for sequence production corresponds to thespeed limit for sequence perception --four per second.

The letter sequence study does not prove that the speed limit for lettersequence production is caused by the speed limit for letter sequenceperception. It may be that the speed limit is imposed by characteristics ofthe vocal apparatus. However, in another study Rosenbaum (1987) found the samefour per second speed limit for production of errorless finger tap sequences.The speed limit for finger tap sequence production is likely to be a perceptualrather than a motor limit because we know that people can produce finger tapsat rates much higher than four per second. Pianists, for example, can dotrills (alternating finger taps) at rates which are far faster than four persecond. Further evidence of the perceptual basis of the finger tap sequencespeed limit would be provided by studies of finger tap sequence perception.When a subject produces a sequence of finger taps he or she is producing asequence of perceptions of pressure at the finger tips. A perceptualexperiment where a pressure is applied to the tip of different fingers insequence should show the four per second speed limit. Subjects should havedifficulty identifying the order of finger tip pressures when the sequenceoccurs at a rate faster than four per second.

Confounding Levels.

It is not always easy to find clear-cutcases of behavioral speed limits that correspond to equivalent perceptual speedlimits. Most behavior involves the control of many levels of perceptionsimultaneously. People control higher level perceptions (like sequences) whilethey are controlling lower level perceptions (like transitions). This can leadto problems when interpreting behavioral speed limits. For example, Rosenbaum(1983) presents some finger tapping results that seem to violate the four persecond speed limit for sequence perception. When subjects tap with two handsthey can produce a sequence of at least 8 finger taps per second. But each tapis not necessarily a separate event in a sequence. Some pairs of taps seem tooccur at the rate at which sequences are experienced as events. A sequence offinger taps is an event in the same sense that the sequence of muscle tensionsthat produce a finger tap is an event; the order of the components of thesequence cannot be perceived. These finger tap events are then unitarycomponents of the sequence of finger tap perceptions. The fact that certainpairs of finger taps are produced as events rather than ordered sequences isindicated by the order errors made at each point in the finger tap sequence.Order errors are greater for the fast pairs than for the slower pairssuggesting that the order of the fast pairs is not under control.

Changing Perception Can Change Behavior.

The relationship between perception and behavior can be seen when a personlearns to perform a task by controlling a new perceptual variable. An exampleof this can be seen in simple pursuit tracking tasks. In the typical trackingtask the target moves randomly. When, however, a segment of target movement isrepeated regularly the subject's tracking performance improves markedly (Pew,1966). According to the hierarchical control model, control is improvedbecause the repeated segment of target movement can be perceived as apredictable event. With the random target the subject must wait to determinetarget position at each instant in order to keep the cursor on target. Withthe repeated target, the subject controls at a higher level. keeping a cursormovement event matching a target movement event. The fact that the subject isnow controlling a higher level perception (an event rather than aconfiguration) is evidence by the longer reaction time when responding to achange in target movement. When controlling the target-cursorconfiguration the subject responds almost immediately to changes in targetposition. When controlling target-cursormovement events it takes nearly 1/2 second to respond to a change to anunexpected target movement pattern.

An experiment by Robertson and Glines (1985) also shows improvedperformance resulting from changed perception. Subjects in the Robertson andGlines study performed a learning task where the solution to a computerizedgame could be perceived at several different levels. Subjects who were able tosolve the game showed three distinct plateaus in their performance. The levelof performance, as indicated by reaction time measurements, improved at eachsucceeding plateau. Because the same outputs (keypresses) were produced ateach level of performance, each performance plateau were taken as evidence ofthat the subject was controlling a different perceptual variable.

Behavior/Perception Correlations.

Few psychologists would be surprised by the main contention of this paper:that there is an intimate relationship between perception and behavior.However, most models of behavior assume that the nature of this relationship iscausal: behavior is guided by perception. This causal model provides no reasonto expect a relationship between the structure of perception and behavior: nomore than there is to expect a relationship between the structure of computerinput and output. This does not mean that there might not be such arelationship; it is just not demanded by the causal model.

The control model integrates perception and behavior with a vengeance.Behavior is no longer an output but, rather, a perceptual input created by thecombined effects of the actor and the environment. Behavior is perception inaction. From this point of view, behavioral skills are perceptual skills.Thus, it is not surprising to find some indication of a correlation betweenbehavioral and perceptual ability (Keele, Pokorny, Corcos and Ovry, 1985).Keele and his colleagues have found that the ability to produce regular timeintervals between actions is correlated with ability to perceive theseintervals. These correlations are fairly low by control theory standards butthey are expected if the production of regular time intervals involves controlof the perception of these intervals.

Conclusion

This report has presented evidence that human behavior involves control ofa hierarchy of perceptual variables. The behavior of other organisms is likelyto involve control of a similar hierarchy of perceptions (Plooij and van de Rift-Plooij,1990). A model of hierarchical control shows how studies of perception andbehavior provide evidence about the nature of control from two differentperspectives. Perceptual studies provide information about the ability toperceive potentially controllable consequences of actions. Behavioral studiesprovide information about the ability to produce desired consequences. Thefactors that influence the ability to perceive the consequences of actionshould also influence the ability to produce them. In both cases we learnsomething about how organisms control their own perceptions.

The hierarchical control model shows that limitations on the ability toproduce behavior may reflect limitations on the ability to perceive intendedresults. The speed at which a person can produce an errorless sequence ofevents, for example, is limited by the speed at which the order of these eventscan be perceived. But not all skill limitations are perceptual limitations.Controlled (perceived) results are produced, in part, by the outputs of thebehaving organism. The ability to produce certain outputs can limit theability to control certain perceptions. For example, it is impossible toperceive oneself lifting a 300 pound barbell until the muscles have beendeveloped to the point that they are able to generate the output forcesnecessary to control this perception.

Perception and behavior are typically treated as two completely separatephenomena. Perception is input: behavior is output. But the concept ofcontrol as the behavior of perception suggests that this separation isartificial. Perception and behavior are just two sides of the process ofcontrol. In order to understand how this process works it will be necessary tounderstand how organisms perceive (perception) and how they act to affect theirperceptions (behavior). Studies of perception and behavior should become anintegral part of the study of a single phenomenon, control.

References

Albus, J. Brains, behavior and robotics. Petersborough, NH: ByteBooks

Arbib, M. (1972) The metaphorical brain. New York: Wiley

Bryan, W. L. and Harter, N. (1899) Studies on the telegraphic language: Theacquisition of a hierarchy of habits. Psychological Review, 6, 345-375

Davis, W. J. (1976) Organizational concepts in the central motor network ofinvertebrates. In R. M. Herman, S. Grillner, P.S.G. Stein, & D. Stuart(Eds.) Neural control of locomotion, New York: Plenum (p. 265)

Greeno, J.G. and Simon, H.A. (1974) Processes for sequence production.Psychological Review, 81, 187-197

Herrigel, E. (1971) Zen in the art of archery. New York: Vintage

Hubel, D. H. and Wiesel, T. N. (1979) Brain mechanisms of vision. In J.M.Wolfe (Ed.) The mind's eye: Readings from Scientific American, New York:Freeman (pp. 40-52)

Keele,S.W., Cohen, A. & Ivry, R. I. (1990) Motor programs: Concepts andissues. In M. Jeannerod (Ed.) Attention and performance XIII: Motorrepresentation and control. New Jersey: Erlbaum. (pp. 77-110)

Keele,S.W., Pokorny, R.A.,Corcos,D.M. & Ivry, R. I. (1985) Doperception and production share common timing mechanisms: A correlationalanalysis. Acta Psychologica, 60, 173-191

Kline, R. (1983) Comment on Rosenbaum et al. Hierarchical control of rapidmovement sequence. Journal of Experimental Psychology: Human Perception andPerformance, 9, 834-36.

Kolers, P.(1972) The illusion of movement. In R. Held and W. Richards (Eds)Perception: Mechanisms and models, San Francisco:Freeman.

Lashley, K.S. (1951) The problem of serial order in behavior. In L.A.Jeffress (Ed.) Cerebral mechanisms in behavior. New York: Wiley

Marken, R.S. (1986) Perceptual organization of behavior: A hierarchicalcontrol model of coordinated action. Journal of Experimental Psychology: HumanPerception and Performance, 12, 254-60.

Marken, R. S. (1988) The nature of behavior: Control as fact and theory,Behavioral Science, 33, 196-206

Marken, R. S. (1989) Behavior in the first degree.In W. Hershberger (Ed)Volitional Action: Conation and control, Amsterdam: North-Holland

Marken, R. S. (1990) Spreadsheet analysis of a hierarchical control systemmodel of behavior, Behavior Research Methods, Instruments, & Computers, 22,349 -359.

Marken, R. S. and Powers, W. T. (1989) Levels of intention in behavior. InW. Hershberger (Ed.) Volitional Action: Conation and Control, Elsevier SciencePublishers: North-Holland.

Martin, J. G. (1972) Rhythmic (hierarchical) versus serial structure inspeech and other behavior. Psychological Review, 79, 487-509

Miller, G. A. and Heise, G. A. (1950) The trill threshold, Journal of theAcoustical Society of America, 22, 637-638

Palmer, S. E. (1977) Hierarchical structure in perceptual representation.Cognitive Psychology, 9, 441-474

Pew, R. W. (1966) Acquisition of hierarchical control over the temporalorganization of a skill. Journal of Experimental Psychology, 71, 764-771

Plooij, F.X. and van de Rift-Plooij,H.H.C.(1990) Developmental transitions as successive reorganizations of acontrol hierarchy. American Behavioral Scientist, 34, 67-80

Povel, D-J(1981) Internal representation of simple temporal patterns. Journal ofExperimental Psychology: Human Perception and Performance, 7, 3 -18

Powers, W. T. , Clark, R. K. and McFarland, R. L. (1960) A general feedbacktheory of human behavior: Part II. Perceptual and Motor Skills, 11, 309-323

Powers, W. T. (1973) Behavior: The control of perception. New York: deGruyter

Powers, W. T. (1979) The nature of robots: Part 4: A closer look at humanbehavior. Byte, August, 309-323

Powers, W. T. (1989) Living control systems. Gravel Switch, KY: CSGPublishing

Powers, W. T. (1990) A hierarchy of control. In R. J. Robertson and W. T.Powers (Eds.) Introduction to modern psychology: The control-theory view, CSG Publishing:

Robertson, R. J. and Glines, L.A. (1985) The phantom plateau returns.Perceptual and Motor Skills, 61, 55-64

Robertson, R. J. and Marken, R. S. (1990) Perception: Input to the controlfunction. In

R. J. Robertson and W. T. Powers (Eds.) Introduction to modern psychology:The control-theoryview, CSG Publishing: Gravel Switch, Ky.

Rosenbaum, D.A. (1987) Hierarchical organization of motor programs. InS.P.Wise (Ed) Higher brain functions: Recent explorations of the brain'semergent properties, New York:Wiley

Rosenbaum, D.A., Weber, R.J. Hazelett, W. M and Hundorff, V. (1986) Theparameter remapping effect in human performance: Evidence from tongue twistersand finger fumblers. Journal of Memory and Language, 25, 710-725

Sperling, G. (1960) The information available in brief visualpresentations. Psychological Monographs, 74 (Whole No. 11)

Sternberg, S., R.L. Knoll, S. Monsell, and C.E. Wright (1989) Motorprograms and hierarchical organization in the control of rapid speech. In M.Jeannerod (Ed.) Attention and performance XIII: Motor representation andcontrol. New Jersey: Erlbaum. (pp. 3-55)

Sternberg, S., R.L. Knoll, D.L. Turock (1990) Hierarchical Control in theexecution of action sequences. Phonetica, 45, 175 197

Warren, R. M. (1974) Auditory temporal discrimination by rained listeners,Cognitive Psychology, 6, 237-256

Warren, R.M., Obusek, C.J., Farmer, R.M. and Warren, R.P. (1969) Auditorysequence: Confusion of patterns other than speech or music. Science, 164, 586-587

Figure 1. Perceptual Control Hierarchy (after Powers, 1989, p 278)

Figure 2. Lower level runaway response to mouse-cursorpolarity reversal.