Abstract
Choice can be driven both by rewards and stimuli that signal those rewards. Under certain conditions, pigeons will prefer options that lead to less probable reward when the reward is signaled. A recently quantified model, the Signal for Good News (SiGN) model, assumes that in the context of uncertainty, signals for a reduced delay to reward reinforce choice. The SiGN model provides an excellent fit to previous results from pigeons and the current studies are the first to test a priori quantitative predictions. Pigeons chose between a suboptimal alternative that led to signaled 20% food and an optimal alternative that led to 50% food. The duration of the choice period was manipulated across conditions in two experiments. Pigeons strongly preferred the suboptimal alternative at the shorter durations and strongly preferred the optimal alternative at the longer durations. The results from both experiments fit well with predictions from the SiGN model and show that altering the duration of the choice period has a dramatic effect in that it changes which of the two options pigeons prefer. More generally, these results suggest that the relative value of options is not fixed, but instead depends on the temporal context.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Imagine making repeated choices between two options that provide delayed reward with the same probability. Choice of one option leads to information about whether the reward is coming, but choice of the other option does not. You are more likely to prefer the informative option, and preference for information is also seen in other species (e.g., Bode et al., 2023; Hursh & Fantino, 1974). We also know that, all else being equal, humans and other animals prefer options that give higher probabilities of reward (e.g., Herrnstein & Loveland, 1975; Shanks et al., 2002). Predictions become less clear, however, when the option providing information is the one that provides less probable reward. In this case, choosing the informative option lowers the obtained reward, a behavior that has been variously called suboptimal choice (e.g., Spetch et al., 1990), paradoxical choice (e.g., Ajuwon et al., 2023), or costly curiosity (Rodriguez Cabrero et al., 2019).
This “suboptimal” choice has now been extensively studied in animals (see Dunn et al., 2024; McDevitt et al., 2016; Vasconcelos et al., 2018; Zentall, 2016, for reviews). Some animals have been shown to seek information even when that choice results in considerably less food (Dunn & Spetch, 1990; Hinnenkamp et al., 2017; Kendall, 1974; Spetch et al., 1994). For example, pigeons and starlings show extreme suboptimal preference in a choice between 20% and 50% reinforcement (e.g., McDevitt et al., 2022; Stagner & Zentall, 2010; Vasconcelos et al., 2015). In that procedure and in the present work (Fig. 1), outcomes are signaled on the 20% food alternative and not signaled on the 50% food alternative. Thus, completion of a choice schedule on the suboptimal (20%) alternative immediately leads to an informative terminal-link stimulus during the delay. That is, one stimulus is presented on trials ending with food and a different stimulus is presented on trials ending without food. Completion of a choice schedule on the optimal (50%) alternative also leads to immediate presentation of one of two stimuli, but they are not correlated with the food and no food outcomes. Overall, the suboptimal option provides information but less frequent food, while the optimal option provides no information but more frequent food.
Recently, suboptimal choice has been the focus of renewed interest and a growing variety of theoretical perspectives (e.g., Ajuwon et al., 2023; Anselme, 2022, 2023; Daniels & Sanabria, 2018; González et al., 2020; Iigaya et al., 2016; McDevitt et al., 2016; Orduña & Alba, 2020; Vasconcelos et al., 2015; Zentall, 2016). Moreover, the tendency to choose options that provide signals for reward (noninstrumental information seeking) is increasingly of interest in neuroscience, cognitive science, and reinforcement learning (Blanchard et al., 2015; Bromberg-Martin & Monosov, 2020; FitzGibbon et al., 2020; Liew, Embrey, & Newell, 2023b; Rodriguez Cabrero et al., 2019).
One approach to understanding suboptimal choice, the Signal for Good News (SiGN) model (McDevitt et al., 2016), is an extension of the Delay Reduction Hypothesis (Squires & Fantino, 1971), which posits that stimuli that signal a reduction in waiting time to food reward become conditioned reinforcers. This hypothesis successfully describes choice between different delays to food but was not specified to account for choice in probabilistic procedures (i.e., when there is uncertainty about whether food will occur). Dunn and Spetch (1990) postulated that when outcomes are delayed and uncertain, signals for food provide extra conditioned reinforcement. Dunn et al. (2024) quantified the SiGN model and found that it provided an excellent fit to the results of 33 existing publications (128 data points) on pigeons and starlings. A unique feature of this model is that it has no free parameters and therefore quantitative predictions can be generated based on procedural variables alone. The current experiments provide the first test of quantitative a priori predictions.
In the SiGN model, temporal parameters are critical because stimuli signaling a reduction in delay to reward function as conditioned reinforcers. Other things being equal, longer delays to food (terminal-link schedules shown in Fig. 1) are more likely to generate preference for the suboptimal alternative in pigeons (McDevitt et al., 2018; Spetch et al., 1990, 1994). Moreover, preference for information about rewards also increases as a function of delay to reward in other species, including humans (e.g., Cunningham & Shahan, 2018; Iigaya et al., 2016, 2020; Liew, Embrey, Navarro, et al., 2023a, and see review by Dunn et al., 2024).
Although there is some suggestion in the literature that the duration of the choice phase (initial link) may influence suboptimal choice, this evidence, as noted by Cunningham and Shahan (2018, 2020), is quite limited. Dunn and Spetch (1990) found that pigeons made fewer suboptimal choices when initial links were longer than a single peck, but preference varied considerably between birds. Pisklak et al. (2019) found less suboptimal choice when 25 pecks were required than when a single peck was required, which is suggestive of temporal effects because 25 pecks take longer to complete. In one of three conditions tested, Zentall et al. (2017) found more suboptimal choice with a 1-s than 20-s choice phase. However, their study was designed to investigate precommitment and included an additional link prior to the choice that differed between the two groups. Thus, the specific effect of the choice duration could not be independently evaluated. In sum, there are indications from the literature that initial-link duration is important, but conclusive evidence for an effect on suboptimal choice is lacking. Our model makes the a priori prediction that manipulating the duration of the choice period should not only alter suboptimal choice but should produce a complete reversal of pigeons’ preferences between two options.
The present experiments are unique in that they systematically assess how initial-link duration influences suboptimal choice and directly test novel quantitative predictions of the SiGN model. Demonstrating temporal context effects on suboptimal choice would provide an important connection to other examples of lawful relationships between temporal factors and behavior. For example, in operant conditioning, time to reinforcement is a critical determinant of reinforcer effectiveness with longer delays producing less effective reinforcement (e.g., Chung & Herrnstein, 1967; Mazur, 1988, 2000). Preference for a signaled reinforcement schedule over an equivalent unsignaled schedule has been found to decline with initial-link duration (Alsop & Davison, 1986; Hursh & Fantino, 1974). However, in those procedures, there was no cost associated with preference for an informative alternative. Temporal relationships are also important in Pavlovian (e.g., Gallistel & Gibbon, 2000; Miller & Barnet, 1993) and reinforcement learning (e.g., Ludvig et al., 2012).
Despite evidence that temporal factors are important for many behaviors, few models of suboptimal choice currently include temporal parameters, and the SiGN model is the only one to make a priori quantitative predictions about the effect of temporal variables. For example, the contrast explanation of suboptimal choice (Stagner & Zentall, 2010) focuses on differences in probability which provides no obvious predicted effect of temporal factors. The ∆−Σ hypothesis (González et al., 2020) is also based on probabilities with no provision for temporal parameters, although it was recently revised to account for some temporal effects (Macías et al., 2024). Firmly establishing temporal effects in suboptimal choice would indicate that the inclusion of temporal variables is a necessary feature of a successful model of suboptimal choice. Cunningham and Shahan (2018, 2020) attribute the absence of temporal factors in some models (e.g., Daniels & Sanabria, 2018; Zentall, 2016) to the limited and inconsistent evidence of initial-link effects and explicitly note that the literature would benefit from additional studies exploring the effects of initial-link duration.
The present experiments manipulate initial-link schedules to assess the effect of temporal context on pigeons’ suboptimal choice. The studies provide a direct test of the SiGN model prediction that increasing the duration of the choice phase reduces preference for the suboptimal alternative. Example calculations are in Appendix A, code for the SiGN model is on the Open Science Framework (Dunn et al., 2022), and an online calculator to generate predictions can be found online (https://jpisklak.shinyapps.io/SiGN_Calc/). The SiGN model predicts that the manipulation of choice duration will produce a complete reversal of preference.
Experiment 1
The initial-link schedule was a fixed ratio (FR) 1 in one condition and a variable interval (VI) 30-s in a comparison condition. The SiGN model predicts exclusive preference for the suboptimal alternative with an FR 1 schedule and strong preference for the optimal alternative with a VI 30-s schedule.
Method
Subjects
The subjects were ten adult pigeons with experience in concurrent chains and simple discrimination procedures and were cared for in accordance with the Guide for the Care and Use of Laboratory Animals (National Research Council, 2011). They were maintained at approximately 85% of their free-feeding weights by grain obtained during experimental sessions and immediate postsession feedings when necessary. The pigeons were housed in individual cages under a 12-hr light/dark cycle, with water and grit freely available. Bird 822 became ill and was removed from the experiment approximately halfway through the second condition, so data analyses include only the first condition for that bird.
Apparatus
Two operant chambers (approximately 360 mm wide, 320 mm long, and 350 mm high) were used. Three circular translucent response keys, 25 mm in diameter, were mounted on the front panel 260 mm above the floor and 72.5 mm apart. The center key was never used in these experiments. Each side key could be illuminated from the rear by standard IEE 28-V 12-stimulus projectors. A 28-V 1-W miniature lamp, located 87.5 mm above the center response key, provided general chamber illumination for the duration of each session, except during blackout periods as noted below. Directly below the center key and 95 mm above the floor was an opening (57 mm high by 50 mm wide) that provided access to a solenoid-operated grain hopper filled with mixed grain. When activated, the food hopper was raised for 5 s and illuminated from above with white light by a 28-V 1-W miniature lamp. A computer and a MED-PC interface, located in an adjacent room, controlled experimental events.
Procedure
Pretraining
Prior to beginning the experiment, each bird received pretraining for two to three sessions during which key pecks to the stimuli used in the experiment were reinforced according to a fixed-ratio (FR) schedule. To ensure that each subject was reliably pecking all stimuli before starting the experiment, the schedule was gradually increased from FR 1 to FR 20.
Training
An overview of the procedure is shown in Fig. 1. The suboptimal alternative was presented on one side key and consisted of a black horizontal line stimulus that, when chosen, was replaced with a color terminal-link stimulus (e.g., green or white keylight) that remained illuminated for 20 s. One terminal-link stimulus (e.g., white) was presented with a probability of .2 and was followed by a 5-s access to the food hopper. The other stimulus (e.g., green) was presented with a probability of .8 and was followed by 5-s termination of the houselight (blackout). Overall, the suboptimal alternative ended with food 20% of the time and the color of the terminal-link stimuli signaled which outcome would occur.
The optimal alternative was presented on the other side key and consisted of a black horizontal line stimulus that, when chosen, was replaced with a color terminal-link stimulus (e.g., yellow or red keylight) that remained illuminated for 20 s. Both terminal-link stimuli appeared equally often (p = .5) and half the time were followed by 5-s access to the food hopper and half the time by blackout. Thus, the optimal alternative ended with food 50% of the time, and the terminal-link stimuli did not differentially signal the outcomes.
The stimulus locations were constant (green and white on the left response key, yellow and red on the right), but the side associated with each alternative was counterbalanced across subjects, so that the optimal alternative was presented on the left for half of the birds and the right for the others. A 5-s intertrial interval separated each trial. Each session consisted of a combination of forced-exposure and choice trials. A forced-exposure trial consisted of the presentation of a single alternative (i.e., the initial-link stimulus on either the right or the left response key). Each block of three trials consisted of two forced-exposure trials (one suboptimal alternative, one optimal alternative) and one choice trial. The order of the trial types was randomized for each block, and sessions ended after a total of 30 trials or after 40 min, whichever occurred first.
Groups and initial-link schedules
The birds were randomly separated into two groups of five birds each. The initial-link schedule differed for the two groups. For one group, completion of an FR 1 schedule was required to enter a terminal link. That is, a single peck to an initial-link stimulus replaced it with a terminal-link stimulus as described above. For birds in the other group, completion of a VI 30 schedule was required to enter a terminal link. The VI operated using a single timer as in McDevitt and Williams (2001) and Roper and Zentall (1999). Thus, for birds in the VI 30 condition, the first peck following completion of the single timer determined which alternative was chosen. This use of a single timer, as opposed to the more common concurrent VI timers, is preferable because it removes the incentive to switch between alternatives (also known as changeover responses) that occurs with concurrent timers (see Pierce & Cheney, 2017, p. 295, for a description of this effect). Frequent switching behavior skews preference toward indifference and requires the addition of a changeover delay as a corrective measure. A single timer eliminates the need for a changeover delay. In addition, the use of a single timer reduces the discrepancy between the programmed and obtained schedule values that can occur with extreme preference levels and potentially bias preference. Thus, the single timer was used to eliminate these unintended influences on responding that might have complicated the interpretation of changes in preference as a function of changes in initial-link duration.
After 16 sessions (Phase 1), the horizontal line stimuli were replaced with squares, the initial-link schedules were switched for the two groups, and training continued for an additional 30 sessions (Phase 2).
Statistical analyses
For both Experiments 1 and 2, graphical and statistical analyses were conducted with R software (Version 4.3.2) using the tidyverse (Version 2.0.0), RColorBrewer (Version 1.1.3), nlme (Version 3.1.163), rsq (Version 2.6), and effsize (v0.8.1) packages (Neuwirth, 2022; Pinheiro & Bates, 2000; R Core Team, 2023; Torchiano, 2020; Wickham et al., 2019; Zhang, 2023). Data and R code are publicly available on the Open-Science Framework (McDevitt et al., 2023).
Within- (schedule) and between-subject (order) effects were analyzed using multilevel linear modeling fit with maximum likelihood estimation. Inverse Bayes factors (BF10) are provided to show how much support each main effect and interaction adds to the model. Differences in model r-squared values (∆R2M) are similarly provided to convey the importance of the included terms.
Results
Choice proportions for the suboptimal alternative were calculated using the last three sessions of each condition. Figure 2 shows the development of preference. Overall, the pigeons showed strong preference for the suboptimal alternative (M = .92, SD = .10) with an FR 1 initial link and strong preference for the optimal alternative with a VI 30-s schedule (M = .20, SD = .20). There was a significant main effect of schedule on suboptimal choice, χ2(1) = 35.95, p < .001, BF10 > 150, ∆R2M = .85. There was no main effect of order, χ2(1) = 0.08, p = .778, BF10 = 0.24, ∆R2M = .001; and no interaction, χ2(1) = 0.23, p = .630, BF10 = 0.26, ∆R2M = .002.
The alternative chosen on the first peck was recorded for birds with the VI initial link at the end of the second phase. The mean choice proportion (all pecks) for those subjects was .20 (SD = .23) and the mean for the first pecks was .43 (SD = .32).
Terminal-link responding on the suboptimal alternative showed consistent evidence that the terminal-link signals were discriminable, with the relative rate of responding on the signal for food exceeding .90 for all birds in both conditions.
Experiment 2
In Experiment 2, all pigeons started in a baseline condition with an intermediate initial-link duration and then were split into two groups for the second condition, one with a shorter choice phase and the other with a longer duration. The initial- and terminal-link values were selected so the SiGN model predicted symmetrical shifts from approximate indifference in the baseline condition to preference for the suboptimal alternative for one group and preference for the optimal alternative for the other group.
Method
Subjects
The subjects consisted of ten pigeons, eight of which also participated in Experiment 1.
Birds 6125 and 392 replaced Bird 822 (the sick bird removed from Experiment 1) and Bird 6 (who was deceased). The pigeons were housed and maintained as described in Experiment 1.
Apparatus
The experimental chambers and equipment described in Experiment 1 were also used in Experiment 2.
Procedure
Pretraining
Prior to beginning the experiment, each bird received the same pretraining experience described in Experiment 1.
Training
The general procedure was as described in Experiment 1 and shown in Fig. 1, but with the following changes to the stimuli and reinforcement schedules. A black ✕ and yellow and red keylights were presented on the left response key, and a black ✕ and blue and white keylights were presented on the right response key. The probability was .2 for entry to both the yellow and blue terminal links and .8 for entry to both the red and white terminal links. In the first condition, the initial-link schedule was the same for all 10 birds and consisted of a single VI 4.75-s timer and the terminal-link was a fixed-time 8-s schedule. This phase continued for 17 sessions, after which the birds were randomly divided into two groups of five. For one group, the initial link was shortened to a VI 1.7-s schedule and for the other it was lengthened to a VI 35-s schedule. The black ✕ initial-link stimuli were replaced with squares in the second phase, which continued for an additional 27 sessions.
Results
Choice proportion calculations again used the last three sessions of each condition and Fig. 3 shows the development of preference. When the choice schedule was VI 4.75, choices favored the suboptimal alternative (M = .65, SD = .30) with considerable variability across subjects. This variability is unsurprising given that this schedule was chosen to approximate indifference, allowing idiosyncratic preferences to emerge.
In the second phase, subjects switched to the VI 1.7-s schedule all strongly preferred the suboptimal alternative (M = .96, SD = .03), and subjects switched to the VI 35-s schedule all strongly preferred the optimal alternative (M = .06, SD = .06). Thus, once the value of the two alternatives diverged, variability decreased sharply.
Two paired t tests and one Welch independent t test were conducted to compare pairwise combinations of schedule types (Welch, 1938). The p values were adjusted for multiple comparisons using the method by Holm (1979), and Hedges’s g effect sizes with corresponding 95% confidence intervals (Borenstein et al., 2009). Within-subject comparison of the VI 4.75 and VI 1.7-s schedules indicated a significant difference; t(4) = −3.10, p = 0.036, g = −1.24, 95% CI [−2.47, −0.01]. Comparison between the VI 4.75 and VI 35-s schedules also showed a significant difference; t(4) = 4.27, p = 0.026, g = 1.39, 95% CI [0.34, 2.45]. Between-subject comparison of the VI 1.7-s and 35-s schedules was also statistically significant with a large effect; t(5.36) = 29.41, p < 0.001, g = 16.8, 95% CI [8.87, 24.73]. First peck data showed the same pattern of initial-link effects (see Appendix B for data and analysis). As in Experiment 1, the relative response rate on the terminal-link signal for food on the suboptimal alternative exceeded .90 for all birds in all conditions.
General discussion
Both experiments show clear effects of temporal context on suboptimal choice, demonstrating that choice of the suboptimal alternative reliably decreases with increases in initial-link duration. These results, together with evidence of an effect of terminal-link duration (McDevitt et al., 2018; Spetch et al., 1990, 1994) confirm that temporal variables critically determine suboptimal preference. These changes in preference are unlikely to be due to artifacts in the arrangement of the VI schedules because the single timer eliminated inadvertent reinforcement for switching between alternatives and reduced the discrepancy between programmed and obtained schedule values.
The first peck measurement demonstrates that the effect of initial-link duration is evident on the first pecks, not just overall choice, and this effect is consistent for every subject. This supports the notion that the relative value of the two alternatives is altered by changes in temporal context and is not due to the dynamics associated with additional time available for responding.
An examination of the first peck data in the longest initial-link conditions (VI 30 and VI 35) is particularly interesting, as these conditions provided the greatest opportunity for preference to shift after the initial response. These data (Table 1) show that some subjects’ choices became more optimal after the first peck. Moreover, the three subjects tested in both conditions were consistent in the degree to which their first peck proportions compared to the overall choice proportions. The first response may be more sensitive to the conditioned reinforcement that underlies suboptimal choice. Laude et al. (2014) reported that individual differences in suboptimal choice were correlated with a measure of “impulsivity” derived from a hyperbolic-delay discounting procedure. Further research might explore whether individual differences in first pecks with long initial links correlate with measures of “impulsivity.”
The striking effect of initial-link duration on suboptimal choice shown here is consistent with the role of temporal variables in associative learning and may reflect a more general phenomenon in which the value of a signal for reward depends on the temporal context in which it is embedded. These results are consistent with the findings of other procedures which found effects of temporal context on preference for information (Alsop & Davison, 1986; Hursh & Fantino, 1974). Further research might systematically explore how temporal context affects other forms of non-instrumental information seeking, a topic that has generated considerable recent interest within neuroscience and human decision making (Bromberg-Martin & Monosov, 2020; FitzGibbon et al., 2020; Liew, Embrey, & Newell, 2023b; Rodriguez Cabrero et al., 2019)
The inverse relationship between initial-link duration and suboptimal choice observed here supported the a priori quantitative predictions of the SiGN model (Dunn et al., 2024). According to this model, both the relative conditioned reinforcement and the relative rate of primary reinforcement shift toward the optimal alternative with longer initial links. Figure 4 shows the SiGN model’s predicted values against obtained choice proportions from both experiments. The correlation between predicted and observed values was very strong, r = .96, t(3) = 5.80, p = .010, and the intercept (b0) and slope (b1) did not significantly differ from 0 and 1, respectively, b0 = −0.14, t(3) = −1.05, p = .370; b1 = 1.20, t(3) = 0.96, p = .408.
The SiGN model predicts choice by considering the combined impact of primary and conditioned reinforcement. However, a different approach to explaining choice that also considers temporal factors is the information-theoretic model (Cunningham & Shahan, 2018). From that perspective, suboptimal choice arises when (1) the suboptimal terminal-link stimuli convey more information about when food will occur, and (2) the delay to terminal-link stimuli is much smaller than the delay to food when a choice is made, thereby biasing animals to base decisions on the temporal informativeness of terminal-link stimuli rather than food rate. This model predicts an increase in optimal choice with increases in initial-link duration by altering the competition between temporal information that favors the suboptimal alternative and relative rate of food that favors the optimal alternative. By adjusting the parameter that modulates bias for using information to make decisions (m) and sensitivity to primary reinforcement (b), the information-theoretic model can provide a good fit to the data presented here (P. Cunningham, personal communication, October 9, 2023). However, doing so requires parameter values that differ considerably from those obtained in Cunningham and Shahan’s (2018) analysis of suboptimal choice data. This highlights a limitation of models with free parameters—namely, a challenge in offering decisive a priori predictions. Nevertheless, the information-theoretic model provides a versatile account of choice in concurrent-chains procedures that corresponds nicely with a cognitive framework valuing information as an explanatory variable.
With the exception of the information-theoretic model, other models of suboptimal choice do not address initial-link duration, which is surprising given that temporal context is a central concept in understanding behavior (e.g., Balsam et al., 2010; Ludvig et al., 2012; Molet & Miller, 2014). Our experiments highlight the need to incorporate temporal variables in models of suboptimal choice. Uncovering mechanisms underlying suboptimal choice in animals adds to our understanding of how organisms respond to reward uncertainty, which may have implications for human behaviors such as risky choice and gambling (e.g., Matthews et al., 2023; Zentall, 2023).
In a clear parallel to suboptimal choice in pigeons, humans and other animals seek advance information about uncertain rewards, sometimes foregoing rewards, paying money, or enduring pain to obtain information even when it does not affect the outcome (e.g., Bennett et al., 2016; Blanchard et al., 2015; Bode et al., 2023; Rodriguez Cabrero et al., 2019). Chasing of prediction error has been proposed as a mechanism of suboptimal choice (Zhu et al., 2017), and the neural circuitry underlying prediction errors and information seeking has been a focus of considerable research (e.g., Bromberg-Martin et al., 2024; Brydevall et al., 2018). Delays between choice and reward can increase preference for information in humans (e.g., Iigaya et al., 2020) suggesting that temporal factors are important. However, the importance of choice duration has been overlooked in the information-seeking literature. Our study shows that altering the choice duration has a dramatic effect and changes which alternative pigeons prefer. This demonstration of a critical role for choice period duration suggests that temporal context warrants further exploration in understanding information seeking.
References
Ajuwon, V., Ojeda, A., Murphy, R. A., Monteiro, T., & Kacelnik, A. (2023). Paradoxical choice and the reinforcing value of information. Animal Cognition, 26(2), 623–637. https://doi.org/10.1007/s10071-022-01698-2
Alsop, B., & Davison, M. (1986). Preference for multiple versus mixed schedules of reinforcement. Journal of the Experimental Analysis of Behavior, 45(1), 33–45. https://doi.org/10.1901/jeab.1986.45-33
Anselme, P. (2022). The optimality of “suboptimal” choice: A psycho-evolutionary perspective. In M. Krause, K. Hollis, & M. Papini (Eds.), Evolution of learning and memory mechanisms (pp. 193–209). Cambridge University Press. https://doi.org/10.1017/9781108768450.014
Anselme, P. (2023). Exploratory search: Information matters more than primary reward. Animal Behavior and Cognition, 10(4), 366–383. https://doi.org/10.26451/abc.10.04.05.2023
Balsam, P. D., Drew, M. R., & Gallistel, C. R. (2010). Time and associative learning. Comparative Cognition & Behavior Reviews, 5, 1–22. https://doi.org/10.3819/ccbr.2010.50001
Bennett, D., Bode, S., Brydevall, M., Warren, H., & Murawski, C. (2016). Intrinsic valuation of information in decision making under uncertainty. PLOS Computational Biology, 12(7), 1–21. https://doi.org/10.1371/journal.pcbi.1005020
Blanchard, T. C., Hayden, B. Y., & Bromberg-Martin, E. S. (2015). Orbitofrontal cortex uses distinct codes for different choice attributes in decisions motivated by curiosity. Neuron, 85(3), 602–614. https://doi.org/10.1016/j.neuron.2014.12.050
Bode, S., Sun, X., Jiwa, M., Cooper, P. S., Chong, T.T.-J., & Egorova-Brumley, N. (2023). When knowledge hurts: Humans are willing to receive pain for obtaining non-instrumental information. Proceedings of the Royal Society B: Biological Sciences, 290(2002), 20231175. https://doi.org/10.1098/rspb.2023.1175
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. John Wiley & Sons, Ltd.
Bromberg-Martin, E. S., Feng, Y., Ogasawara, T., White, J. K., Zhang, K., & Monosov, I. E. (2024). A neural mechanism for conserved value computations integrating information and rewards. Nature Neuroscience, 27(1), 159–175. https://doi.org/10.1038/s41593-023-01511-4
Bromberg-Martin, E. S., & Monosov, I. E. (2020). Neural circuitry of information seeking. Current Opinion in Behavioral Sciences, 35, 62–70. https://doi.org/10.1016/j.cobeha.2020.07.006
Brydevall, M., Bennett, D., Murawski, C., & Bode, S. (2018). The neural encoding of information prediction errors during non-instrumental information seeking. Scientific Reports, 8, 6134. https://doi.org/10.1038/s41598-018-24566-x
Chung, S.-H., & Herrnstein, R. (1967). Choice and delay of reinforcement. Journal of the Experimental Analysis of Behavior, 10(1), 67–74. https://doi.org/10.1901/jeab.1967.10-67
Cunningham, P. J., & Shahan, T. A. (2018). Suboptimal choice, reward-predictive signals, and temporal information. Journal of Experimental Psychology: Animal Learning and Cognition, 44(1), 1–22. https://doi.org/10.1037/xan0000160
Cunningham, P. J., & Shahan, T. A. (2020). Delays to food-predictive stimuli do not affect suboptimal choice in rats. Journal of Experimental Psychology: Animal Learning and Cognition, 46(4), 385–397. https://doi.org/10.1037/xan0000245
Daniels, C. W., & Sanabria, F. (2018). An associability decay model of paradoxical choice. Journal of Experimental Psychology: Animal Learning and Cognition, 44(3), 258–271. https://doi.org/10.1037/xan0000179
Dunn, R. M., & Spetch, M. L. (1990). Choice with uncertain outcomes: Conditioned reinforcement effects. Journal of the Experimental Analysis of Behavior, 53(2), 201–218. https://doi.org/10.1901/jeab.1990.53-201
Dunn, R. M., Pisklak, J. M., McDevitt, M. A., & Spetch, M. L. (2022). The signals for good news (SiGN) model data and code repository. https://doi.org/10.17605/OSF.IO/39QTJ
Dunn, R. M., Pisklak, J. M., McDevitt, M. A., & Spetch, M. L. (2024). Suboptimal choice: A review and quantification of the signal for good news (SiGN) model. Psychological Review, 131(1), 58–78. https://doi.org/10.1037/rev0000416
FitzGibbon, L., Lau, J., & Murayama, K. (2020). The seductive lure of curiosity: Information as a motivationally salient reward. Current Opinion in Behavioral Sciences, 35, 21–27. https://doi.org/10.1016/j.cobeha.2020.05.014
Gallistel, C., & Gibbon, J. (2000). Time, rate, and conditioning. Psychological Review, 107(2), 289–344. https://doi.org/10.1037/0033-295X.107.2.289
González, V. V., Macías, A., Machado, A., & Vasconcelos, M. (2020). The ∆−Σ hypothesis: How contrast and reinforcement rate combine to generate suboptimal choice. Journal of the Experimental Analysis of Behavior, 113(3), 591–608. https://doi.org/10.1002/jeab.595
Herrnstein, R. J., & Loveland, D. H. (1975). Maximizing and matching on concurrent ratio schedules. Journal of the Experimental Analysis of Behavior, 24(1), 107–116. https://doi.org/10.1901/jeab.1975.24-107
Hinnenkamp, J. E., Shahan, T. A., & Madden, G. J. (2017). How suboptimal is suboptimal choice? Journal of the Experimental Analysis of Behavior, 107(1), 136–150. https://doi.org/10.1002/jeab.239
Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65–70. https://www.jstor.org/stable/4615733.
Hursh, S., & Fantino, E. (1974). An appraisal of preference for multiple versus mixed schedules. Journal of the Experimental Analysis of Behavior, 22(1), 31–38. https://doi.org/10.1901/jeab.1974.22-31
Iigaya, K., Story, G. W., Kurth-Nelson, Z., Dolan, R. J., & Dayan, P. (2016). The modulation of savouring by prediction error and its effects on choice. eLife, 5. https://doi.org/10.7554/eLife.13747
Iigaya, K., Hauser, T. U., Kurth-Nelson, Z., O’Doherty, J. P., Dayan, P., & Dolan, R. J. (2020). The value of what’s to come: Neural mechanisms coupling prediction error and the utility of anticipation. Science Advances, 6(25), eaba3828. https://doi.org/10.1126/sciadv.aba3828
Kendall, S. B. (1974). Preference for intermittent reinforcement. Journal of the Experimental Analysis of Behavior, 21(3), 463–473. https://doi.org/10.1901/jeab.1974.21-463
Laude, J. R., Beckmann, J. S., Daniels, C. W., & Zentall, T. R. (2014). Impulsivity affects suboptimal gambling-like choice by pigeons. Journal of Experimental Psychology: Animal Behavior Processes, 40(1), 2–11. https://doi.org/10.1037/xan0000001
Liew, S. X., Embrey, J. R., Navarro, D. J., & Newell, B. R. (2023a). Comparing anticipation and uncertainty-penalty accounts of noninstrumental information seeking. Decision, 10(3), 247–267. https://doi.org/10.1037/dec0000179
Liew, S. X., Embrey, J. R., & Newell, B. R. (2023b). The non-unitary nature of information preference. Psychonomic Bulletin & Review, 30(5), 1966–1974. https://doi.org/10.3758/s13423-022-02243-5
Ludvig, E. A., Sutton, R. S., & Kehoe, E. J. (2012). Evaluating the TD model of classical conditioning. Learning & Behavior, 40(3), 305–319. https://doi.org/10.3758/s13420-012-0082-6
Macías, A., González, V., Machado, A., & Vasconcelos, M. (2024). Time, uncertainty, and suboptimal choice. Behavioural Processes, 214, 104982. https://doi.org/10.1016/j.beproc.2023.104982
Matthews, J. R., Cooper, P. S., Bode, S., & Chong, T.T.-J. (2023). The availability of non-instrumental information increases risky decision-making. Psychonomic Bulletin & Review, 30(5), 1975–1987. https://doi.org/10.3758/s13423-023-02279-1
Mazur, J. (1988). Estimation of indifference points with an adjusting-delay procedure. Journal of the Experimental Analysis of Behavior, 49(1), 37–47. https://doi.org/10.1901/jeab.1988.49-37
Mazur, J. (2000). Tradeoffs among delay, rate, and amount of reinforcement. Behavioural Processes, 49(1), 1–10. https://doi.org/10.1016/S0376-6357(00)00070-X
McDevitt, M., & Williams, B. (2001). Effects of signaled versus unsignaled delay of reinforcement on choice. Journal of the Experimental Analysis of Behavior, 75(2), 165–182. https://doi.org/10.1901/jeab.2001.75-165
McDevitt, M. A., Dunn, R. M., Spetch, M. L., & Ludvig, E. A. (2016). When good news leads to bad choices. Journal of the Experimental Analysis of Behavior, 105(1), 23–40. https://doi.org/10.1002/jeab.192
McDevitt, M. A., Pisklak, J. M., Spetch, M. L., & Dunn, R. M. (2018). The influence of outcome delay on suboptimal choice. Behavioural Processes, 157, 279–285. https://doi.org/10.1016/j.beproc.2018.10.008
McDevitt, M. A., Pisklak, J. M., Dunn, R. M., & Spetch, M. L. (2022). Forced-exposure trials increase suboptimal choice. Psychonomic Bulletin & Review, 29(4), 1514–1523. https://doi.org/10.3758/s13423-022-02092-2
McDevitt, M. A., Pisklak, J. M., Dunn, R. M., & Spetch, M. L. (2023). Temporal context effects on suboptimal choice data and code repository. https://osf.io/kpxzr
Miller, R. R., & Barnet, R. C. (1993). The role of time in elementary associations. Current Directions in Psychological Science, 2(4), 106–111. https://doi.org/10.1111/1467-8721.ep10772577
Molet, M., & Miller, R. R. (2014). Timing: An attribute of associative learning. Behavioural Processes, 101, 4–14. https://doi.org/10.1016/j.beproc.2013.05.015
National Research Council. (2011). Guide for the care and use of laboratory animals (6th ed.). The National Academies Press. https://doi.org/10.17226/12910
Neuwirth, E. (2022). RColorBrewer: ColorBrewer palettes (R Package Version 1.1-3) [Computer software]. https://CRAN.R-project.org/package=RColorBrewer
Orduña, V., & Alba, R. (2020). The incentive salience of the stimuli biases rats’ preferences in the “suboptimal choice” procedure. Behavioural Processes, 172, 104057. https://doi.org/10.1016/j.beproc.2020.104057
Pierce, W. D., & Cheney, C. D. (2017). Behavior analysis and learning: A biobehavioral approach (6th ed.). Routledge. https://doi.org/10.4324/9781315200682
Pinheiro, J. C., & Bates, D. M. (2000). Mixed-effects models in S and S-PLUS. Springer. https://doi.org/10.1007/b98882
Pisklak, J. M., McDevitt, M. A., Dunn, R. M., & Spetch, M. L. (2019). Suboptimal choice and initial-link requirement. Journal of the Experimental Analysis of Behavior, 112(3), 242–253. https://doi.org/10.1002/jeab.553
R Core Team. (2023). R: A language and environment for statistical computing. R Foundation for Statistical Computing https://www.R-project.org/
Rodriguez Cabrero, J. A. M., Zhu, J., & Ludvig, E. A. (2019). Costly curiosity: People pay a price to resolve an uncertain gamble early. Behavioural Processes, 160, 20–25. https://doi.org/10.1016/j.beproc.2018.12.015
Roper, K. L., & Zentall, T. R. (1999). Observing behavior in pigeons: The effect of reinforcement probability and response cost using a symmetrical choice procedure. Learning and Motivation, 30(3), 201–220. https://doi.org/10.1006/lmot.1999.1030
Shanks, D. R., Tunney, R. J., & McCarthy, J. D. (2002). A re-examination of probability matching and rational choice. Journal of Behavioral Decision Making, 15, 233–250. https://doi.org/10.1002/bdm.413
Spetch, M. L., Belke, T. W., Barnet, R. C., Dunn, R. M., & Pierce, W. D. (1990). Suboptimal choice in a percentage-reinforcement procedure: Effects of signal condition and terminal-link length. Journal of the Experimental Analysis of Behavior, 53(2), 219–234. https://doi.org/10.1901/jeab.1990.53-219
Spetch, M. L., Mondloch, M. V., Belke, T. W., & Dunn, R. M. (1994). Determinants of pigeons’ choice between certain and probabilistic outcomes. Animal Learning & Behavior, 22(3), 239–251. https://doi.org/10.3758/bf03209832
Squires, N., & Fantino, E. (1971). A model for choice in simple concurrent and concurrent-chains schedules. Journal of the Experimental Analysis of Behavior, 15(1), 27–38. https://doi.org/10.1901/jeab.1971.15-27
Stagner, J. P., & Zentall, T. R. (2010). Suboptimal choice behavior by pigeons. Psychonomic Bulletin & Review, 17(3), 412–416. https://doi.org/10.3758/PBR.17.3.412
Torchiano, M. (2020). Effsize: Efficient effect size computation (R Package Version 0.8.1) [Computer software]. https://doi.org/10.5281/zenodo.1480624
Vasconcelos, M., Monteiro, T., & Kacelnik, A. (2015). Irrational choice and the value of information. Scientific Reports, 5(1), 13874. https://doi.org/10.1038/srep13874
Vasconcelos, M., Machado, A., & Pandeirada, J. N. (2018). Ultimate explanations and suboptimal choice. Behavioural Processes, 152, 63–72. https://doi.org/10.1016/j.beproc.2018.03.023
Welch, B. L. (1938). The significance of the difference between two means when the population variances are unequal. Biometrika, 29(3/4), 350–362. https://doi.org/10.2307/2332010
Wickham, H., Averick, M., Bryan, J., Chang, W., McGowan, L. D., François, R., Grolemund, G., Hayes, A., Henry, L., Hester, J., Kuhn, M., Pedersen, T. L., Miller, E., Bache, S. M., Müller, K., Ooms, J., Robinson, D., Seidel, D. P., Spinu, V., ..., Yutani, H. (2019). Welcome to the tidyverse. Journal of Open Source Software, 4(43), 1686. https://doi.org/10.21105/joss.01686
Zentall, T. R. (2023). An animal model of human gambling behavior. Current Research in Behavioral Sciences, 4. https://doi.org/10.1016/j.crbeha.2023.100101
Zentall, T. R. (2016). Resolving the paradox of suboptimal choice. Journal of Experimental Psychology: Animal Learning and Cognition, 42(1), 1–14. https://doi.org/10.1037/xan0000085
Zentall, T., Andrews, D., & Case, J. (2017). Prior commitment: Its effect on suboptimal choice in a gambling-like task. Behavioural Processes, 145, 1–9. https://doi.org/10.1016/j.beproc.2017.09.008
Zhang, D. (2023). Rsq: R-squared and related measures (R Package Version 2.6) [Computer software]. https://CRAN.R-project.org/package=rsq
Zhu, J. Q., Xiang, W., & Ludvig, E. A. (2017). Information seeking as chasing anticipated prediction errors. In G. Gunzelmann, A. Howes, T. Tenbrink, & E. J. Davelaar (Eds.), Proceedings of the 39th annual meeting of the cognitive science society (pp. 3658–3663). Cognitive Science Society https://cogsci.mindmodeling.org/2017/papers/0690/index.html
Author note
Funding provided by Natural Sciences and Engineering Research Council Discovery Grant 2020-03933 to M.L.S. We thank the McDaniel College students in the Spring 2022 and Spring 2023 Psychology of Learning classes for assistance with data collection. We thank Paul Cunningham for helpful discussions about the information-theoretic model. Data and R code are publicly available on the Open-Science Framework (https://osf.io/kpxzr/)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
McDevitt, M.A., Pisklak, J.M., Dunn, R.M. et al. Temporal context effects on suboptimal choice. Psychon Bull Rev (2024). https://doi.org/10.3758/s13423-024-02519-y
Accepted:
Published:
DOI: https://doi.org/10.3758/s13423-024-02519-y