Impulsive Choice and Pre-Exposure to Delays: IV. Effects of Delay- and Immediacy-Exposure Training Relative to Maturational Changes in Impulsivity

Abstract

Impulsive choice describes preference for smaller, sooner rewards over larger, later rewards. Excessive delay discounting (i.e., rapid devaluation of delayed rewards) underlies some impulsive choices, and is observed in many maladaptive behaviors (e.g., substance abuse, gambling). Interventions designed to reduce delay discounting may provide therapeutic gains. One such intervention provides rats with extended training with delayed reinforcers. When compared to a group given extended training with immediate reinforcers, delay-exposed rats make significantly fewer impulsive choices. To what extent is this difference due to delay-exposure training shifting preference toward self-control or immediacy-exposure training (the putative control group) shifting preference toward impulsivity? The current study compared the effects of delay- and immediacy-exposure training to a no-training control group and evaluated within-subject changes in impulsive choice across 51 male Wistar rats. Delay-exposed rats made significantly fewer impulsive choices than immediacy-exposed and control rats. Between-group differences in impulsive choice were not observed in the latter two groups. While delay-exposed rats showed large, significant pre- to post-training reductions in impulsive choice, immediacy-exposed and control rats showed small reductions in impulsive choice. These results suggest that extended training with delayed reinforcers reduces impulsive choice, and that extended training with immediate reinforcers does not increase impulsive choice.

Keywords:

delay-exposure training, impulsive choice, delay discounting, impulsivity, lever press, rat

Method

Subjects

Subjects were 52 naïve male Wistar rats (Harlan Laboratories, Indianapolis, IN), approximately 21 days old at intake. One rat assigned to the IE group was excluded from analysis because of a persistent side bias. This study was conducted in cohorts of four to eight rats per cohort over the course of approximately 22 months. All rats were individually housed in a humidity and temperature controlled animal colony room that operated on a 12-hr light:dark cycle (lights on at 7:00 am). Following 7 days of ad-libitum food access, rats were gradually restricted to 85% of their growth curve free-feeding weights. Unless otherwise noted, all rats were maintained at their 85% weight for the duration of the study. Free access to water was available in the home cage. Experimental sessions were conducted at the same time each day and supplemental food was delivered approximately 2 hrs post session. All work was conducted under a protocol approved by the Institutional Animal Care and Use Committee at Utah State University.

Apparatus

Nineteen operant chambers (Med Associates, St. Albans, VT), each housed within a sound-attenuating cubicle with a ventilation fan, were used. Two low-profile retractable levers were positioned on the front wall (6.5 cm above the grid floor) of the chamber. A food dispenser was positioned outside the chamber that delivered 45-mg pellets (Bio-Serv, Frenchtown, NJ) to a receptacle centered between the two front-wall levers (2.5 cm above the grid floor). An identical lever was centered on the rear-wall of the chamber (6.5 cm above the grid floor). A 28-V cue light was placed above each lever and a white-noise generator was positioned in the upper right corner of the rear wall (13 cm above the grid floor). During lever training, an 8 oz plastic water bottle was mounted outside the chamber. The spout entered the chamber to the left of the rear-wall cue light (4 cm above the grid floor).

Locomotor activity was assessed with a circular corridor apparatus constructed of two PVC pipes (30.5 cm in height, 66.0 and 45.7 cm, for the diameter of the outside and inside walls, respectively; see, e.g., Perry et al., 2008; Piazza, Deminiere, Le Moal, & Simon, 1989). Four infrared sensors were mounted within the walls of the corridors (5.1 cm above the grid floor), and were equidistant from each other such that their placement formed four quadrants (i.e., one sensor at 0°, 90°, 180°, and 270°). The top of the apparatus was covered with a removable sheet of clear Plexiglas. The room was equipped with a white-noise generator.

Procedures

depicts the order of experimental conditions and the median age of rats during each condition. Briefly, locomotor activity was assessed followed by lever-press training. Next, rats completed amount-discrimination training and a pre-training impulsive-choice assessment. Rats were then assigned to the DE, IE, or CONT group. While DE and IE rats completed their respective training, CONT rats remained fallow in their home cages but were otherwise treated identically as DE and IE rats; that is, CONT rats were maintained at their 85% free-feeding weight, handled, and fed in the same manner as the other groups. After 120 days, all rats completed amount-discrimination training followed by the post-training impulsive-choice assessment. The details of each phase are outlined below.

Open in a separate window

Locomotor assessment.

Prior to food deprivation, locomotor activity was assessed using the procedures outlined by Perry et al. (2005). Rats were placed in the circular corridor apparatus for two 45-min sessions, and sessions were conducted across two consecutive days. Locomotor counts were defined as an interruption of two adjacent photobeams in succession; breaking the same photobeam twice consecutively was not scored as a locomotor count. A white-noise generator was on for the duration of testing.

Lever training.

Lever training was conducted during overnight sessions; access to water was provided during these sessions. Each session consisted of four 20-trial blocks during which white noise was presented, and each block was separated by a 60-min blackout during which no stimuli were presented. Initially, rats were trained to press the two front-wall levers. Each trial began with the insertion of either the left or right front-wall lever (order pseudorandomly determined). If 55 s elapsed without a response, the cue light above the lever was illuminated for up to 5 s. If the lever was not pressed during the 60-s trial, the lever retracted, the cue light turned off, and one food pellet was delivered. Pressing the lever during the trial delivered one food pellet, retracted the lever, and a new trial was initiated. Training continued until rats pressed the inserted lever on ≥ 90% of the trials in the final two trial blocks. The same procedure was used to train rear-wall lever pressing, the exception being that the consequence of pressing the rear wall was the retraction of that lever and the insertion of one of the front-wall levers. One food pellet was delivered for pressing the front-wall lever. Training continued until rats pressed the rear- and front-wall levers on ≥ 90% of the trials in the final two trial blocks. Throughout the experiment, sessions were conducted at approximately the same time daily (between 9:00 am and 5:00 pm), and individual rats progressed to the next phase after meeting the task-specific progression criteria (if present).

Pre-training amount discrimination.

Amount-discrimination sessions were composed of three, 20-trial blocks, with each block separated by a 7-min blackout. Each block was composed of 6 forced-choice trials followed by 14 free-choice trials. All trials began by activating the light-cued rear-wall lever. When this lever was pressed, either one (forced-choice trials) or two (free-choice trials) front-wall levers were inserted into the chamber and the corresponding cue light(s) illuminated. Pressing either lever once retracted the lever(s), turned the cue light(s) off, and delivered the food amount programmed on the lever—either one or three pellets (lever assignment counterbalanced across rats). An adjusting inter-trial interval (ITI) ensured that a new trial started every 60 s. Failure to respond to a lever within 30 s retracted the lever(s), turned off the cue light(s), and was scored as an omission. Omitted forced-choice trials were repeated. White noise was presented throughout the session during this and all subsequent phases. Sessions ended when all 60 trials were completed or if 2 hrs elapsed. Amount-discrimination training sessions continued until rats selected the three-pellet alternative on ≥ 90% of the trials across two consecutive sessions.

Pre-training impulsive-choice assessment.

Impulsive choice was assessed using a within-session, increasing-delay procedure (e.g., Evenden & Ryan, 1996). Sessions were structured identically to the amount-discrimination sessions, with the exception that the delay to the three-pellet alternative increased across the three successive trial blocks in the following order: 0, 15, 30 s. The one-pellet alternative was always delivered immediately.

Following 6 sessions, all rats completed a single amount-discrimination probe session (i.e., the delay to the three-pellet reward was 0 s throughout the session). This session was conducted to ensure that rats were not habitually responding to avoid the LLR during the second and third trial blocks. After this probe session, rats were returned to the increasing-delay procedure for at least 6 additional sessions and until the following stability criteria were met: 1) ≥ 80% choice of the three-pellet alternative in the 0-s delay block for 5 consecutive sessions, 2) area under the curve (AUC; see Myerson, Green, & Warusawitharana, 2001) in each of the final 5 sessions did not deviate by more than 20% from the mean of these final 5 sessions, and 3) no monotonic increasing or decreasing trend in AUC over the final 5 sessions.

If, during the impulsive-choice assessment, preference for the three-pellet alternative in the 0-s delay block fell below 60% for two consecutive sessions, rats were placed into remedial amount-discrimination sessions (programmed as above and continued until achieving two consecutive days of ≥ 90% choice of the three-pellet alternative). If this failed to re-establish sensitivity to reward amount, two or more sessions were conducted in which only the lever associated with the three-pellet alternative was presented for 60 trials. Subsequently, remedial amount-discrimination sessions were conducted until the aforementioned criterion was met. Thereafter, impulsive-choice sessions continued until the stability criteria were met.

Group assignment.

Because this study was conducted in cohorts, rats were assigned to DE, IE, or CONT groups in a way that minimized between-group differences in pre-training impulsive choice (AUC) and 2-day mean locomotor counts.

DE, IE, and no training.

During DE and IE training sessions, each trial began with the insertion of the rear-wall lever and illumination of the cue light above that lever. For DE rats, a single press retracted the lever and initiated a 17.5-s delay, after which the cue light turned off and two food pellets were delivered. For IE rats, a single response retracted the lever, turned off the cue light, and delivered two food pellets immediately. Two pellets were delivered so the reward amount during exposure training would not match either reward available in the impulsive-choice assessments. For both groups, failure to press the rear-wall lever within 20 s was scored as an omission and omitted trials were repeated. An adjusting ITI ensured a new trial began every 60 s. Sessions ended when the rats completed 80 trials or if 2 hrs elapsed. DE and IE training continued for 120 sessions. Rats in the CONT group were handled, weighed, and treated identically to rats in the DE and IE groups, but were fallow for 120 days. Due to experimenter error, six CONT rats were fallow for an additional 9–32 days; there was no difference in post-training impulsive choice (AUC) for CONT rats that received additional fallow days and those that received 120 days (p = .94).

Post-training amount discrimination.

After DE, IE, or no training, amount-discrimination training sessions were conducted. The procedures and criteria to progress to the next phase were as described above with the exception that the food amounts assigned to the left and right levers during the pre-training amount-discrimination phase were switched. These assignments were unchanged for the remainder of the experiment.

Post-training impulsive-choice assessment.

After rats met the amount-discrimination criteria, impulsive choice was reassessed. Procedures, stability criteria, and remedial sessions (if necessary) were as described above.

Data Analysis

Before conducting statistical analyses, univariate and bivariate normality of variables was assessed as appropriate; univariate normality was tested using the Shapiro-Wilk test. When the data in question significantly differed from a normal distribution, nonparametric tests were used in lieu of their parametric counterpart.

Prior to examining differences in impulsive choice, group differences in lever and exposure training were examined. A Kruskall-Wallis test was used to examine between-group differences in the number of days to meet the lever-training acquisition criteria. Wilcoxon rank-sum tests were used to examine differences between the DE and IE groups on response latencies during the final 5 sessions of exposure training. All rats completed all 80 trials during these final sessions, so no analysis of trials completed was conducted. For all analyses here and below, p values < .05 were considered statistically significant.

Group differences in non-choice dependent measures from the impulsive-choice assessments were also evaluated. Kruskall-Wallis tests were conducted to examine between- and within-group differences on: 1) sessions to meet the amount-discrimination criterion, 2) sessions to stability of LLR choice, 3) omissions, and 4) latencies to press the SSR and LLR levers on forced- and free-choice trials (the latter two measures were averaged over the final 5 sessions). Minimal between-subject variability in pre-training impulsive choice precluded a valid assessment of the trait-like stability of this behavior over time.

The effects of training and maturation on impulsive choice were examined using a generalized linear mixed effects (GzLME) analysis (for similar approaches, see Young, in press; Young, 2017). Of particular interest were the within-group differences in choice from pre- to post-training for CONT rats (maturation effects), differences in choice between the IE and CONT groups at the post-training assessment (to determine if IE training increases impulsivity), and finally, differences in choice between the DE and CONT groups (to determine if DE training increases self-control relative to changes due to maturation). Differences between the DE and IE groups in degree of impulsive choice were assessed for the purpose of evaluating the replicability of previous reports (Renda & Madden, 2016; Stein et al., 2013; Stein et al., 2015). Individual choices at each delay (SSR or LLR, coded as 0 and 1, respectively) across the final 5 sessions of the pre- and post-training impulsive-choice assessments served as the dependent variable in the GzLME analysis. This yielded 210 choices per rat (14 free-choice trials per delay x 3 delays x 5 stable sessions), per assessment. The outcome was specified as binomial to accommodate the binary nature of choice, and a logit link function was used.

Ultimately, the GzLME is the equivalent of a repeated-measures logistic regression. The independent variables included in the model were Assessment (Pre-training/Post-training), Group (DE/IE/CONT), and Delay (0 s/15 s/30 s) all as categorical variables, with all of their interactions; a significant three-way interaction was anticipated due to the nature of the study design (i.e., DE rats should have bigger changes in the likelihood of choosing the LLR from pre- to post-training than IE or CONT rats, and self-control should decrease as the delay to the LLR increased, but to different extents across groups due to training and/or maturation). A random intercept of subject was included in the model. The results were nominally the same whether Delay was entered as a continuous or categorical predictor; thus, for ease of interpretation and facilitating comparisons, the categorical type was chosen. To evaluate the significance of the predictors in a manner similar to obtaining F-statistics in ANOVAs, Wald tests were computed using the Companion to Applied Regression (car) package (Fox & Weisberg, 2002). The necessity of random slope effects of Delay (i.e., the functional equivalent of allowing for individual differences in discounting rates, above and beyond that captured by group-level differences) was subsequently evaluated using a likelihood ratio test. No other random effects were evaluated.

All analyses were conducted in R (R Core Team, 2013). Normality testing was conducted using the nortest package (Gross & Ligges, 2015). GzLME models were fitted using the lme4 package (Bates, Mächler, Bolker, & Walker, 2015), and the lsmeans package (Lenth, 2016) was used to generate contrasts from the GzLME (to examine maturational and/or training effects). All other analyses were conducted using base R functions, except where noted.

Results

By nature of the assignment of subjects to groups, all groups were equivalent on measures of locomotor activity (see ) and pre-training AUC at the start of the experiment (ps ≥ .28). Likewise, there were no between-group differences in the number of days to acquire lever pressing, Kruskal Wallis χ2 (2, N = 51) = .31, p = .86 (see ).

Table 1.

DEIECONTLocomotor counts19.7 (16.6–24.0)19.1 (16.8–21.7)19.8 (15.3–23.4)Days to acquisition criteria2 (2–2)2 (2–2)2 (2–2)Open in a separate window

During DE and IE training, rats in both groups completed all trials. shows individual-subject latencies to respond and omissions during DE and IE training (top and bottom panel, respectively); bars correspond to medians and error bars to IQR. Over the final 5 sessions, DE rats had significantly longer response latencies, W = 226, p = .004, and made significantly more omissions, W = 230, p < .001, than IE rats.

Open in a separate window

shows pre- and post-training data from amount-discrimination and impulsive-choice phases. The median number of sessions to meet the stability criteria are shown, along with omissions and response latencies. No between-group differences in omissions or latencies were statistically significant in the pre- or post-training assessments, although differences in the latencies to respond on smaller-sooner forced-choice trials in the post-training assessment approached significance, χ2 (2, N = 51) = 4.97, p = .08. From pre-to post-training, the only significant within-group non-choice difference was a reduction in the days to meet the amount-discrimination criteria in the IE group, W = 88.5, p = .05. Some response latencies either significantly, or nearly significantly declined from pre- to post-training in the CONT (forced SSR, W = 66, p = .006; forced LLR, W = 75, p = .02; free SSR, W = 49, p = .001) and IE groups (forced SSR, W = 95, p = .09; free LLR, W = 90, p = .06).

Table 2.

Pre-trainingPost-trainingDEIECONTDEIECONTDays to discrimination criteria4 (3–8)4 (3–8)4 (3–5)4 (3–5)3 (2–4)*4 (3–5)Days to stability criteria18 (15–27)15 (14–23)16 (15–18)16 (14–20)17 (14–21)18 (15–24)Omissions0.0 (0.0–0.0)0.0 (0.0–0.0)0.0 (0.0–0.0)0.0 (0.0–0.0)0.0 (0.0–0.5)0.0 (0.0–0.0)Response latency: Forced-choice SSR1.6 (1.4–1.7)1.6 (1.5–2.0)1.8 (1.6–1.9)1.8 (1.4–2.1)1.5 (1.3–1.8)1.4 (1.2–1.6)**Response latency: Forced-choice LLR1.5 (1.4–2.1)2 (1.5–2.4)1.6 (1.3–2)1.4 (1.1–1.7)1.5 (1.1–2.2)1.2 (1–1.5)*Response latency: Free-choice SSR1.7 (1.5–2.0)1.7 (1.3–2.4)1.9 (1.6–2.1)1.6 (1.4–2.2)1.5 (1.3–1.9)1.4 (1.2–1.5)**Response latency: Free-choice LLR1.5 (1.3–1.6)1.4 (1.3–1.8)1.5 (1.3–1.6)1.5 (1.3–2.2)1.3 (1.1–1.5)1.2 (1.1–1.7)Open in a separate window

The left two columns of show individual-subject percent LLR choice across delays in the pre- and post-training impulsive-choice assessments for DE, IE, and CONT groups (top, middle, and bottom panels, respectively). The right column of shows individual-subject and median (± IQR) change in percent LLR choice from pre- to post-training across delays. In the GzLME analysis, the interaction between Assessment, Group, and Delay was significant, χ2(4) = 57.55, p < .0001, as were the majority of the other predictors in the model (see ). This model was improved by allowing the effect of delay to vary across subjects, χ2(5) = 461.57, p < .0001.

Open in a separate window

Table 3.

PredictorChi-SquaredfpGroup3.902.14Assessment753.951< .0001Delay454.732< .0001Group*Assessment180.392< .0001Group*Delay4.204.38Assessment*Delay197.192< .0001Group*Assessment*Delay57.554< .0001Open in a separate window

shows the model-predicted percent LLR choice by delay (± 1 SEM) for all groups in the pre- and post-training impulsive-choice assessment (left and right panels, respectively). In the absence of a universally-agreed upon metric of fit for nonlinear models, the representativeness of the model predictions and the adequacy of the modeling procedure itself is reflected in comparing the group-level estimates in to the individual-subject values in . At the pre-training assessment, all rats showed very low percent LLR choice at both the 15- and 30-s delays, and there were no significant differences between groups at any of the delays (ps > .15); thus, AUC was an adequate dependent measure for evaluating equivalence in choice at baseline.

Open in a separate window

Overall, DE training reduced impulsive choice relative to both IE and CONT rats. Replicating previous studies, DE rats showed significantly greater percent LLR choice than IE rats at both the 15-s (59.76% vs. 8.43%; z = 4.57, p < .0001) and 30-s delays (30.11% vs. 3.48%; z = 2.81, p = .005). The DE rats also showed greater self-control than the CONT rats, although the effects were slightly smaller. This was evidenced by significantly greater percent LLR choice at the 15-s delay (59.76% vs. 20.92%; z = 2.86, p = .004), but a difference that only approached significance at the 30-s delay (30.11% vs. 8.61%; z = 1.75, p =.08). At the 15-s delay, IE training produced a near-significant difference in percent LLR choice relative to CONT rats (8.43% vs. 20.92%, respectively; z = 1.73, p = .08); however, choice at the 30-s delay was unaffected by IE training (3.48% vs. 8.61%; z = 1.08, p =.28).

Lastly, there was evidence of a maturation-related reduction in impulsive-choice in the CONT group. From pre- to post-training, percent LLR choice significantly increased at both the 15-s (from 6.36% to 20.92%; z = 10.90, p < .0001) and 30-s delays (from 0.10% to 8.61%; z = 11.56, p < .0001).

Acknowledgments

This research was supported financially by a grant from the National Institutes of Health: R21 DA042174, awarded to the last author (G. J. Madden). None of the authors have any real or potential conflict(s) of interest, including financial, personal, or other relationships with organizations or pharmaceutical/biomedical companies that may inappropriately influence the research and interpretation of the findings. All authors have contributed substantively to this study and have read and approved the final manuscript. All authors would like to thank Thomas Argyle, Jessica Baird, Kyle Butler, Rowan Crowder, Jacob Goddard, and Michael Williams for their assistance in conducting the experiment.

Footnotes

1As applied to these data, CL effect size is the probability that a randomly selected DE rat will make less impulsive choices than a randomly selected IE rat (Lakens, 2013). CL effect size is robust to violations of normality (see McGraw & Wong, 1992).

2The locomotor assessment served as a precursor for future studies in our lab examining the effects of DE/IE training on subsequent drug self-administration. Because locomotor activity in the circular corridor is predictive of drug self-administration (e.g., Piazza et al., 1989), matching based on this variable ensures that differences in drug responding are not due to differences in baseline locomotor activity. Prior research has found no difference in locomotor behavior (as measured with the circular corridor) between high- and low-impulsive rats (see Perry et al., 2005; Perry et al., 2008).

3AUC is a summary measure of delay discounting, reflecting the area under the stable percent LLR choices made at the range of delays investigated. Thus, higher values of AUC reflect a greater preference for the LLR (i.e., greater self-control)

Contributor Information

C. Renee Renda, Utah State University.

Jillian M. Rung, Utah State University.

Jay E. Hinnenkamp, Utah State University.

Stephanie N. Lenzini, Utah State University.

Gregory J. Madden, Utah State University.