Relative and Absolute Strength of Response as a Function
of Frequency of Reinforcement^1,2

R. J. Herrnstein
Harvard University

The present experiment explores the relative frequency of responding further. In the earlier study is was shown that the output of behavior to each of two keys may be controlled by specific requirements of outputs. Now we are investigating output as a function of frequency of reinforcement. The earlier experiment may be considered a study of differential reinforcement; the present one, a study of strength of response. Both experiments are attempts to elucidate the properties of relative frequency of responding as a dependent variable.

METHOD

Three adult, male, White Carneaux pigeons, maintained at 80% of free-feeding weights, and experimentally naive at the start of the study, were used.

Apparatus

A conventional experimental chamber for pigeons (Ferster & Skinner, 1957) was modified to contain two response-keys. Each key was a hinged, translucent Plexiglas plate mounted behind a hole in the center partition of the chamber. The pigeons pecked at a circular area (diameter = 0.75 inch) of the plate, and a force of at least 15 grams was necessary to activate the controlling circuitry. Any effective response operated an audible relay behind the center position; it has been found that the resulting auditory feedback stabilizes the topography of pecking. Behind each key was a group of Christmas-tree lamps of various colors, each group mounted in such a way that it cast significant amounts of light through only one key. The two keys were 4.5 inches apart (center-to-center) around the vertical midline of the center partition and on a horizontal line about 9 inches from the floor of the chamber. Though a 2-inch-square hole in the center position, 2 inches from the floor, the pigeon occasionally received the reinforcer - 4 second's access to grain.

A masking noise and a low level of general illumination were provided.

Procedure

Preliminary training lasted for two sessions of 60 reinforcements each. During these sessions, a peck to either key was reinforced only when the just-previous reinforcement was for a peck to the other key. This alternating pattern of reinforcement led rapidly to a pattern of responding that consisted of almost perfect alternation between the two keys. The left key was always red; the right, always white.

During the experiment proper, responding to either key was reinforced on a variable-interval schedule. The schedule for one key was independent of the schedule for the other. Thus, at any given moment, reinforcement could be made available on neither key, on one key or the other, or on both keys. A reinforced response to one key had no effect on the programmer that scheduled reinforcements on the other.

The primary independent variable was the mean time interval between reinforcements on each key. These intervals were chosen so that the mean interval of reinforcement for the two keys taken together was held constant at 1.5 minutes.³ The over-all average value of 1.5 minutes was produced by a number of pairs of values for the two keys. The combined frequency of reinforcement from independent variable-interval schedules will be a constant if the values for each of the two keys are chosen according to the hyperbolic relationship:

1/x + 1/y = 1/c;

in which x is the mean interval on one key, y is the mean interval on the other, and c is the combined mean interval for the two keys taken together. The pairs of values used were VI(3) VI(3); VI(2.25) VI(4.5); VI(1.8) VI(9); and VI(1.5) VI(infinity) - i.e., extinction on one of the keys.

During most of the experiment, the pigeons were penalized for switching from one key to the other. Each time a peck to one key followed a peck to the other key, no reinforcement was possible for 1.5 seconds. Thus, the pigeon never got fed immediately after changing keys. When the pigeon switched keys before the 1.5-second period was completed, the period simply started anew. At least two consecutive pecks on a given key were necessary before reinforcement was possible: the first peck to start the period, and the second after it was completed. This penalty for alternation will be referred to as the "change-over delay of 1.5 seconds," or COD(1.5").

The sequence of pairs of values of the variable interval schedules and the number of sessions at each pair of values are shown in Table 1. Key A is the left, red key; Key B is the right, white key. Sessions lasted for 60 reinforcements, which required approximately 90 minutes since the over-all mean interval of reinforcement was always 1.5 minutes. Whether the COD was present or absent is also shown.

RESULTS

Figure 1 shows the relative frequency with which the pigeon pecked on Key A as a function of the relative frequency with which it was reinforced on that key.

Fig. 1. Relative frequency of responding to Key A as a function of relative frequency of reinforcement on Key A, for three pigeons; COD(1.5") is present throughout.

Each point on the graph is a mean of the last five sessions under a given pair of values of the variable-interval schedule. The COD operated on all these sessions; the results without the COD will be given later. The ordinate and abscissa values were calculated by comparable methods. The number of responses (ordinate) or reinforcements (abscissa) on Key A was divided by the total number of responses or reinforcements, respectively. The five last sessions were pooled to make this computation.

The diagonal line with a slope equal to 1.0 in Fig. 1 shows the function that would be obtained if the relative frequency of responding were exactly equal to the relative frequency of reinforcement. The empirical values approximate the theoretical function with a maximum discrepancy of only about 8%. There seems to be no regular pattern to the deviations from the theoretical function.

The absolute rate of responding on each of the keys is shown in Fig. 2. Responses per hour are plotted against reinforcements per hour, for each key separately and for the two pigeons (231 and 055) that had an appreciable range of the independent variable. Data from the same sessions are plotted in Fig. 1 and 2.

Fig. 2. Rate of responding on each key as a function of rate of reinforcement on that key, for two pigeons; COD(1.5") is present throughout.

With one exception (Pigeon 055, Key A, at 40 reinforcements per hour), the points in Fig. 2 approximate a linear function that passes through the origin. It will be shown later that this relation between absolute rate of responding and absolute rate of reinforcement is the simplest one that is compatible with the relative-frequency function presented in Fig. 1.

The number of times a pigeon changed keys depended on the difference in frequency of reinforcement on the two keys. Figure 3 shows this relation for the three pigeons. The abscissa gives the difference, without regard to sign, between per cent of total reinforcement on one key and that on the other. Thus, when the two keys are characterized by equal relative frequencies of reinforcement, the value on the abscissa is 0; when the responding to one key is extinguished, the value is 100, and so on. The ordinate gives simply the average number of times the pigeon switched from Key A to Key B, or vice versa. Once again, the data are from the same sessions that supplied those in Fig. 1.

Fig. 3. Number of alternations between the two keys as a function of the absolute difference between the per cent of reinforcements on each key, for three pigeons; COD(1.5") is present throughout.

It should be noted, however, that in Fig. 3 there are only four values for Pigeons 055 and 231 whereas there were six in Fig. 1 and 2. This is the result of combining the three pairs of variable-interval schedules involving mean intervals of 9 minutes and 1.8 minutes. (See Table 1.) The data at abscissa values of about 70 per cent are, therefore, based on means of 15, instead of 5, sessions. The functions in Fig. 3 are less consistent than those in Fig. 1 and 2, but the frequency of alternations between keys clearly decreases as the two keys are associated with increasingly different relative frequencies of reinforcement.

The relation shown in Fig. 3 is found only when the COD is in operation. Figure 4 shows the frequency of key changes with and without the COD when reinforcement frequency is either equally or unequally distributed between the two keys. The data from Pigeon 231 are omitted from this figure, because this bird was not exposed to any procedure that combined unequal frequencies of reinforcement with no COD. Two facts are evident in Fig. 4.

Fig. 4. Number of alternations between the two keys when the COD was present or absent and when reinforcements were equally or unequally distributed between the two keys, for the two pigeons.

One is that the COD markedly reduces the frequency of alternations between the keys. The other is that unequal reinforcement frequencies on the two keys reduce alternation only when the COD (1.5") is present.

The COD also seems to play a role in the production of the relation shown in Fig. 1, namely, the tendency of the relative frequency of responding to match the relative frequency of reinforcement. Pigeons 055 and 641 were both exposed to procedures in which the COD was absent and the relative frequency of reinforcement on Key A was about 66%. The relative frequencies of responding on Key A were 50% and 56% for the two pigeons, respectively. In both these cases the departures from matching are outside the range of departures obtained when the COD is present. (See Fig. 1.)

DISCUSSION

The major problem posed by the present experiment is to explain the simple correspondence in fig. 1 between the relative frequency of reinforcement and the relative frequency of responding. In a sense, this correspondence is readily explained by the curves in Fig. 2, which suggest that the relation between the absolute rate of responding and the absolute rate of reinforcement is a linear function that passes through the origin. If this relation is represented as p = ke, in which p and e denote the absolute frequencies of pecking and eating, then the simple matching function of Fig. 1 may be expected to follow the form

The constant, k, drops out and the remaining expressions on each side of the equation denote relative frequencies of responding and reinforcement. The equality of these two relative frequencies may thus be regarded as a consequence of a linear relation, of any slope and zero intercept, between the absolute frequencies. Moreover, this relation between the absolute rates of responding and reinforcement is one that is consonant with a plausible view of response strength: Rate of responding is a linear measure of response strength, which is itself a linear function of frequency of reinforcement. The correspondence in Fig. 1 would thereby result from the fact that the behavior on each of the two keys obeys a simple linear rule governing strength of response. According to this point of view, the animals match relative frequency of responding to relative frequency of reinforcement not because they take into account what is happening on the two keys, but because they respond to the two keys independently.

The crucial relation, p = ke, has been asserted before. Skinner (1938, p. 130) has discussed a quantity called the extinction ratio, which is the total number of responses divided by the number of reinforced responses in a fixed-interval schedule of reinforcement.⁴ He presented a small amount of data that indicated that this quantity remained constant as the size of the fixed interval was varied. The constancy of the extinction ratio is merely another form, p/e = k, of the function we find.

Perhaps the greatest vulnerability of the foregoing account lies in its simplicity. If it were true that the rate of responding is so simply related to the frequency of reinforcement, that fact ought to have been well established by now We should expect that behavior in a single-key situation would reveal the same linear relation shown in Fig. 2, and that with all the work done with the single-key problem, the nature of the relation between rate of responding and frequency of reinforcement would be known. Unfortunately, this information is not available. In few studies has the frequency of reinforcement been varied over an adequately wide range. Those which have done so have usually also involved manipulations in other, and possibly contaminating, variables.

A small amount of relevant material is shown in Fig 5. These curves are adapted from earlier studies by Clark (1958), Wilson (1954), and Herrnstein (1955). These three experimenters observed the convention of plotting the independent variable as inter-reinforcement time, rather than frequency of reinforcement. Clark and Wilson used rats (Wilson used fixed-interval, instead of variable-interval, schedules); Herrnstein used pigeons.

Fig. 5. Data from previous experiments replotted to show rate of responding as a function of frequency of reinforcement.

Rate of responding clearly increases with frequency of reinforcement. In these one-response situations, however, we do not obtain the linear function with zero intercept that was shown in Fig. 2. The relation suggested by Fig. 5 has downward concavity. Even if this concavity is taken to represent nothing more than a natural ceiling on the rate of responding, the function is still inappropriate, because the intercept is greater than zero.

Perhaps a more relevant comparison can be made with some data of Findley (1958), who devised a modification of concurrent scheduling not unlike the present procedure. A pigeon responds to a key and is reinforced on a variable-interval schedule. The two variable-interval schedules are independent, just as in the present study. The difference between Findley's procedure and ours is in the character of the switching response. Switching required a peck on a second key in Findley's experiment, whereas in ours the pigeon had only to move over. In the present experiment, the discriminative stimuli for both schedules were concurrently visible; in Findley's , only one was present at a time, but the other was always available via a switching response. Figure 6 shows the relation between absolute rate of responding and absolute frequency of reinforcement obtained by Findley.

Fig. 6. Data from Findley's experiment (1958) replotted to show rate of responding as a function of frequency of reinforcement. Numbers on curves refer to individual subjects (pigeons).

Findley did not keep the total frequency of reinforcement constant as he varied the average inter-reinforcement interval associated with the two colors. Pigeons 5 and 6 had a constant value of 6 minutes in one color, and values ranging from 2 to 20 minutes in the other. The graphs are for the varying component. Responding in the other component was not, however, constant. Reynolds (1961) has demonstrated a similar kind of interaction in an ordinary multiple schedule. For Pigeons 2 and 4, the schedules were varied in both components. Only for Pigeon 5 does the function appear linear with an intercept of zero. For the three other pigeons, the pattern was the same as in Fig. 5: The relation is either concave downwards or linear with an intercept greater than zero.

Our results suggest that a relative-frequency function with a slope of less than 1.0 over part of the range would have been obtained if it were not for the COD. The precise correspondence between relative frequency of responding and relative frequency of reinforcement broke down when the COD was omitted. When the relative-frequency relationship has a slope of less than 1.0, then the absolute-frequency relationship must either be concave downwards or linear with a positive intercept. Thus, the present experiment shows excellent matching, on the one hand, and atypical absolute rate functions, on the other, probably because of the COD.

It remains to be explained why the COD has the effect of bringing the empirical points closer to the perfect-matching function. The data in Fig. 4 show that the COD greatly reduces the frequency of alternation between the two keys. Without the COD, switching is reinforced by the occasions when the first peck to a key produces food; with the COD, these occasions never occur. In a sense, then, switching is a third operant in the situation and is extinguished by the COD. The abundance of switching with no COD would tend to make the frequency of responding to the two keys more nearly equal than they would be if switching were not being reinforced. The reduction of switching by the COD probably does not, however, explain why the absolute rate of responding in the present experiment follows the simple linear function. At best, one would expect the absolute rate to behave the way it does in single-key experiments. Single-key experiments, including Findley's version of the concurrent schedule, yield functions between absolute rate of responding and absolute frequency of reinforcement that predict non-matching functions between relative frequency of responding and relative frequency of reinforcement. A way of characterizing this finding is to say that in the single-key situation, the animal responds too much as the low frequencies of reinforcement or too little at the high. Thus, the curves in Fig. 5 and 6 have intercepts greater than zero or are concave downwards. The same would apparently have been true in the present experiment had it not been for the COD. Why the COD has this effect is intuitively, if not scientifically, obvious. With a COD, two things are likey to happen; both follow from the fact that once the animal has switched to a key, it is likely to stay there for at least the duration of the COD. First of all, if the tendency to respond to a key is low, then the COD will probably push the tendency even lower because switching to the key calls for not one but a number of responses. Second, if the tendency is high, then the total number of pecks to the key will probably be increased because each switch to the key guarantees a number of responses. These presumed effects would change a function that is concave downwards or linear with a positive intercept toward a function that is linear with an intercept of zero. As an analogy, separating the two keys spatially may have effects similar to those of the COD. If the two keys were far apart, the animal would probably be less likely to go to the less lucrative key than if only a small distance were involved, and responding to the more lucrative key would be increased. A COD whose duration is longer than the 1.5 seconds used here might give a matching function with a slope greater than 1.0 and absolute-rate functions that are concave upwards or linear with intercepts less than zero.

The suggestion of the present discussion is that the surprisingly precise correspondence between relative frequency of responding and relative frequency of reinforcement arises from the function relating absolute frequency of responding and absolute frequency of reinforcement . When this function is linear with an intercept of zero, matching is found. In single-key situations, this linear relation is not obtained; and it is also not obtained under concurrent schedules unless some additional procedural factor reduces the pigeon's tendency to over-respond at low frequencies of reinforcement and under-respond at high. The COD is such a procedural factor; but others, such as distance between keys or effort involved in the response, ay also be satisfactory. The duration of the COD may or may not be critical in the effect is has on the slope of the relative-frequency function. If a broad range of durations of the COD all give approximately perfect matching, then is seems correct to say that the concurrent procedure is a good one for studying absolute, as well as relative, strength of responding. In single-key situations, the rate of responding is not very sensitive to frequency of reinforcement. This insensitivity probably weakens our interest in the concept of strength of response. It may be that the concept can be given significant empirical support in multiple-key situations.

SUMMARY

A two-key, concurrent procedure involving a variable-interval schedule on each key was used. The value of the mean interval on each key was varied over a range fro 1.5 to 9 minutes, but the total frequency of reinforcement for the two keys taken together was held constant. The pigeon was penalized for alternating in response between the two keys by making reinforcement impossible for 1.5 seconds after every alternation. It was found that the relative frequency of responding on a given key closely approximated the relative frequency of reinforcement on that key.

REFERENCES

Clark, F. C. The effect of deprivation and frequency of reinforcement on variable-interval responding. J. exp. Anal. Behav., 1958, 1, 221-228.

Ferster, C. B., and Skinner, B. F. Schedules of reinforcement. New York: Appleton-Century-Crofts, 1957.

Findley, J. D. Preference and switching under concurrent scheduling. J. exp. Anal. Behav., 1958, 1, 123-144.

Herrnstein, R. J. Behavioral consequences of the removal of a discriminative stimulus associated with variable-interval reinforcement. Unpublished doctoral dissertation, Harvard Univer., 1955.

Herrnstein, R. J. Some factors influencing behavior in a two-response situation. Trans. N. Y. Acad. Sci., 1958, 21, 35-45.

Reynolds, G. S. Relativity of response rate and reinforcement frequency in a multiple schedule. J. exp. Anal. Behav., 1961, 2 179-184.

Skinner, B. F. The behavior of organisms. New York: D. Appleton Century Co., 1938.

Wilson, M. P. Periodic reinforcement interval and number of periodic reinforcements as parameters of response strength. J. comp. Physiol. Psychol., 1954, 47 51-56.

Received November 14, 1960.

FOOTNOTES