Brain stimulation reward

Brain stimulation reinforcement

Early studies on the motivational effects of brain stimulation addressed two primary questions:

1. Which brain sites can be stimulated to produce the perception of reward? and
2. Which drugs influence the response to stimulation and via what mechanism?

Investigation of the brain reward circuitry reveals that it consists of a distributed, multi-synaptic circuit that determines both BSR and natural reward function. The natural drives that motivate and shape behavior reach the reward circuitry trans-synaptically through the peripheral senses of sight, sound, taste, smell, or touch. Experimentally-induced BSR more directly activates the reward circuitry and bypasses transduction through peripheral sensory pathways. For this reason, electrical brain stimulation provides a tool for identifying the reward circuitry within the central nervous system with some degree of anatomical and neurochemical specificity. Studies involving these two forms of laboratory reward showed stimulation of a broad range of limbic and diencephalic structures could be rewarding as well as implicated the dopamine-containing neurons of the mesolimbic dopamine system in motivational function. The motivational effect of intracranial self-stimulation varies substantially depending on the placement site of the surgically implanted electrode during electrical stimulation, and animals will work to stimulate different neural sites depending on their current state. Often, animals that work to initiate brain stimulation will also work to terminate the stimulation.

Relationship to natural rewards and drives

The relationship between BSR and natural rewards (e.g. food, water and copulation) has long been debated, and much of the early research on BSR is focused on their respective similarities and differences. BSR is facilitated through the same reinforcement pathway activated by natural rewards. Self-stimulation can exert robust activation of central reward mechanisms due to more direct action than natural rewards, which initially activate peripheral nerves. BSR to the medial forebrain bundle (MFB) through either electrical or chemical means activates key components of the reward pathway also activated by natural rewards. When specific regions of the hypothalamus are electrically stimulated, it elicits reward-related behaviors such as eating, drinking, or copulation responses.

Natural rewards are associated with a state of deprivation from unmet needs or desires (e.g., hunger). These states drive instinctual, motivated behaviors like food consumption. It has been argued that this is not the case with BSR, as it does not meet an intrinsic survival-based need. BSR also notably lacks an established neural representation in memory that naturally facilitates the learning of reward expectancy. Both of these effects lead to diminished response rate for BSR in the early trials of a series; however, experiments have also shown that extinguished behavior can be quickly reinstated by a priming stimulation that refreshes the short-term association involved in reward expectancy. Studies on BSR indicate that reinforcing brain stimulation may activate the natural pathways associated with natural drives as well as stimulate the reinforcement pathways that are usually activated by natural rewards.

Strength of drive

Rats will perform lever-pressing at rates of several thousand responses per hour for days in exchange for direct electrical stimulation of the lateral hypothalamus. Multiple studies have demonstrated that rats will perform reinforced behaviors at the exclusion of all other behaviors. Experiments have shown rats will forgo food to the point of starvation in exchange for brain stimulation or intravenous cocaine when both food and stimulation are offered concurrently for a limited time each day. Rats will also cross electrified grids to press a lever, and they are willing to withstand higher levels of shock to obtain electrical stimulation than to obtain food.

Satiation

Satiation experiments in rats have revealed that BSR does not produce satiety. Olds demonstrated that this lack of satiation associated with BSR allows animals to self-stimulate to sheer exhaustion and that satiation is dependent on the location of the electrical stimulation. In a 48-hour satiation test, rats with hypothalamic electrodes self-stimulated to exhaustion and showed no intrinsic satiation tendencies, whereas telencephalic electrodes showed radical slowing of self-stimulation after 4 to 8 hours. The insatiability of BSR is closely related to the strength of drive. While a natural reward, like food, is met with a feeling of being full (satiety), BSR does not have a comparable correlate. This allows for BSR to be experienced indefinitely, or in the case of ICSS, until exhaustion.

Addiction

Addiction is a chronic brain disorder consisting of compulsive drug-taking and seeking that is maintained despite detrimental effects on various aspects of life including health, relationships, and work. Laboratory procedures can establish compulsive self-administration habits of seeking and ingesting that qualify as addictive behaviors. Rodents and non-human primates have been shown to work in a compulsive manner to receive intravenous injections of stimulants, and when access to the drugs is not limited, they will self-administer drugs to the point of severe weight loss and death. Similar to self-administration behavior, responding for intracranial brain stimulation has a highly compulsive component characteristic of an addicted state. BSR is hypothesized to be so effective in establishing compulsive habits due to its more direct activation of the reward pathway, bypassing transmission through sensory pathways in response to natural rewards. Delayed reinforcement following a response for BSR decreases how strongly this behavior is reinforced and to what extent it continues. A delay of one second, for example, between a lever-press and reward delivery (stimulation) can reduce response levels. BSR offers insights into the neural circuitry involved in reinforcement and compulsive behavior.

Anatomy of reward

Mapping and lesion studies on BSR were designed to determine the location of reward-relevant neurons as well as determine the signal pathways that are directly affected by brain stimulation. The site of intracranial self-stimulation leads to substantially different behavioral characteristics. Sites along the length of the medial forebrain bundle (MFB) through the lateral and posterior hypothalamus, the ventral tegmental area (VTA), and into the pons are associated with the strongest reward effects of stimulation.

Lateral hypothalamus

The lateral hypothalamus is a portion of the hypothalamus, and brain stimulation to this area at the level of the medial forebrain bundle produces the highest response rates and subsequently the highest reward potency in rodents. Lesions in this region or along its boundary cause a loss of positive drive-reward behaviors as well as all other operant drive behaviors.

Medial forebrain bundle

The medial forebrain bundle (MFB) is the location of the most frequently investigated brain stimulation reward sites, and it is composed of a complex bundle of axons projecting from the basal olfactory regions and the septal nuclei. MFB is not the sole anatomical substrate responsible for reinforcing brain stimulation; however, it is the main tract for the ascending dopamine fibers, and it functions to relay information from the VTA to the nucleus accumbens. The rewarding effect of MFB stimulation is mediated via the activation of the mesocorticolimbic dopamine system.

Mesolimbic pathway

The mesolimbic pathway connects the VTA to the nucleus accumbens. The nucleus accumbens is located in the ventral striatum and integrates information from cortical and limbic brain structures to mediate behaviors the reinforce reward. It is a major target for the dopaminergic projections from the VTA, a group of neurons located close to the midline on the floor of the midbrain. The VTA is the origin of dopaminergic cell bodies that comprise the mesocorticolimbic dopamine system.

BSR has been shown to result in the release of dopamine within the nucleus accumbens, which also occurs in response to natural rewards such as food or sex.

Raphe nucleus

The raphe nucleus, which is the origin of most of the serotonergic innervation of the forebrain, is one of the most potent brain areas for inducing self-stimulation. Its capacity in this regard is equivalent to the medial forebrain bundle (MFB) and ventral tegmental area (VTA).

Indirect activation

Electrophysiological data suggest stimulation of the MFB or VTA does not directly activate dopaminergic neurons in the mesolimbic reward pathway. These data suggest BSR is facilitated by initial excitation of descending, myelinated neurons, which then activate the ascending, unmyelinated neurons of the VTA. Excitatory, cholinergic inputs to the VTA are thought to play a role in this indirect activation, but the neuroanatomical components of this circuit have yet to be fully characterized.

Intracranial self-stimulation (ICSS) procedures

Initial training

Since the initial demonstration of BSR by Olds and Milner, experiments in rodents record ICSS responding to quantify motivation to receive stimulation. Subjects undergo stereotactic surgery to permanently implant either a monopolar or bipolar electrode to the desired brain region. Electrodes are connected to a stimulating apparatus at the time of the experiment.

The first portion of an ICSS experiment involves training subjects to respond for stimulation using a fixed-ratio 1 (FR-1) reinforcement schedule (1 response = 1 reward). In experiments involving rats, subjects are trained to press a lever for stimulation, and the rate of lever-pressing is typically the dependent variable. In ICSS studies using mice, a response wheel is usually used instead of a lever, as mice do not consistently perform lever-pressing behaviors. Each quarter turn of the response wheel is recorded and rewarded with stimulation. The rewarding stimulus in BSR experiments is typically a train of short-duration pulses separated by interval pulses, which can be manipulated experimentally using the independent variables of stimulation amplitude, frequency, and pulse duration.

Establishing minimum effective current

The amplitude (current) of stimulation determines the population of neurons being activated by the implanted electrode. In certain approaches, this is adjusted for every subject due to minor variability in electrode placement, and therefore a slightly different population of affected neurons. Following FR1 training, it is typical to establish a minimum, threshold current that produces sufficient levels of ICSS responding (about 40 responses per minute). This is called a discrete-trial current intensity procedure. Each discrete trial consists of non-contingent stimulation at a certain amplitude followed by a brief window during which the animal can respond for more stimulation. Effective currents for BSR elicit responding above a certain rate (3 out of 4 trials, for example). The lowest current the animal responds sufficiently to is deemed the minimum effective current. This is done at a constant frequency, typically at the higher end of the frequency range employed in ICSS studies (140–160 Hz).

Within-subject study design is often implemented to help eliminate variability introduced by electrode placement. Between-subject study design requires rigorous histologic verification of electrode placement to ensure consistency between experimental groups. Subjects with imperfect electrode placement require a higher simulation amplitude to activate the reward circuitry and produce ICSS responding. Subjects with ideal anatomical placement will respond at lower stimulation amplitudes. This corrective process is limited, however, since increasing the population of activated neurons can result in off-target activation of neighboring circuitry. This is often culminated in undesired motor side effects upon stimulation, due to the adjacency of the MFB to the internal capsule, a bundle of axons carrying descending motor information to the brainstem. Inadvertent stimulation of these axons can lead to motor output such as movement of the head or paw twitching.

Frequency-rate responding

At a constant minimum effective current, ICSS responding is recorded over a series of trials, which vary in stimulation frequency. Each trial consists of a short priming phase of non-contingent stimulation, a response phase where responses are recorded and rewarded with stimulation, and a short time-out phase where responses are not recorded and no stimulation is delivered. This is repeated for a series of 10-15 different ascending or descending frequencies, in 0.05 log-unit increments, which range anywhere from 20 to 200 Hz.

While the amplitude of the stimulation influences which neurons are stimulated, the frequency of stimulation determines the firing rate induced in that neuronal population. Generally, increasing stimulation frequency increases the firing rate in the target population. This is associated with higher ICSS response rates, eventually reaching a maximum level at the maximum firing rate, limited by the refractory properties of neurons.

Other factors

The independent variables of stimulation train and pulse duration can also be varied to determine how each affects ICSS response rates. Longer train durations produce more vigorous responding up to a point, after which rate of responding varies inversely with train length. This is due to lever-pressing for additional stimulation before the previously earned train has finished.

The reinforcement schedule can also be manipulated to determine how motivated an animal is to receive stimulation, reflected by how hard they are willing to work to earn it. This can be done by increasing the number of responses required to receive a reward (FR-2, FR-3, FR-4, etc.) or by implementing a progressive-ratio schedule, where the number of required responses continually increases. The number of required responses increases for each trial until the animal fails to reach the required number of responses. This is considered the "break-point" and is a good indication of motivation related to reward magnitude.

Curve-shift analysis

Stimulation intensity, pulse duration, or pulse frequency can be varied to determine dose-response functions ICSS responding using curve-shift analysis. This approach generally resembles traditional pharmacological dose-response curve where the frequency of stimulation, rather than the dose of a drug, is examined. This method allows for quantitative analysis of reward-modulating treatments on response rates in comparison to baseline conditions. Lower stimulation frequencies fail to sustain ICSS responding at a probability above chance. Response rates increase rapidly over a dynamic range of stimulation frequencies as the frequency increases, until a maximum response rate is reached. Changes in the rate of response over this range reflects changes in the magnitude of the reward. Rate-frequency, rate-intensity, or rate-duration functions make inferences about the potency and efficacy of stimulation, as well as elucidate how drugs alter the rewarding impact of stimulation.

Curve-shift analysis is often used in pharmacological studies to compare baseline response rates to those following drug administration. The maximum response rate during baseline conditions is typically used to normalize data in a frequency-rate curve to a maximum control rate (MCR). More specifically, the number of responses for any given trial is divided by the highest number of responses recorded in a baseline condition trial, which is then multiplied by 100. In an experimental condition, if the MCR falls below 100% at the highest stimulation frequencies, it is thought to reflect an impacted capability or motivation to respond, potentially induced by a drug with sedative or aversive properties. Shifts above 100% of the MCR indicate improved ability or motivation to respond, potentially induced by a drug with rewarding or stimulant properties.

Sensitivity of the neural circuitry to the rewarding properties of stimulation is assessed by analyzing left- or right-shifts in the M50, or the frequency at which 50% of the maximum number of responses was recorded. Reaching 50% of the MCR at a lower frequency is characteristic of a left-shift in the frequency-rate curve and sensitization of the reward circuitry to stimulation. An increase in the M50 indicates that a greater stimulation frequency was required to reach 50% of the MCR, and the reward circuitry has been desensitized by the experimental manipulation. Another way of analyzing the frequency-rate curve between control and experimental conditions is to do a linear regression through the ascending data points in a plot of raw data (which has not been normalized to the MCR). The point where y=0, or the x-intercept, is called the threshold frequency or theta zero (θ₀). This is the frequency at which ICSS response rates are equal to 0 (and any frequency above this will theoretically elicit ICSS responding).

Modulation with drugs

Several major drug classes have been studied extensively in relation to ICSS behavior: monoaminergic drugs, opioids, cholinergic drugs, GABAergic drugs, as well as a small number of drugs from other classes. These studies generally compare ICSS responding at baseline and following drug administration. Typically, the frequency-rate approach is used to determine changes in the M50 or θ_0.

Drugs with increased addiction liability generally decrease the stimulation threshold for ICSS responding, while drugs with aversive properties generally increase the stimulation threshold to achieve ICSS responding. These studies provide insight into the specific neurochemical mechanisms involved in the facilitation of BSR, and how reward perception can be modulated by pharmacologically altering the activity of specific neurotransmitter systems. Pharmacological manipulation of these systems can have either a direct or indirect effect on the activity of the reward circuitry. Understanding drug-specific effects on ICSS response thresholds has helped elucidate how different neurotransmitter systems influence the reward circuitry by either potentiating or suppressing sensitivity to rewarding stimulation and influencing motivation to perform reward-associated behaviors.

BSR and drugs of addiction produce their rewarding effects through shared neuroanatomical and neurochemical mechanisms. Studies using lesion, pharmacological, and anatomical mapping of the brain have revealed that many drugs of abuse (e.g. amphetamine, cocaine, opioids, nicotine, etc.) activate the reward circuitry of the MFB, which is similarly activated by stimulation to achieve BSR. The neuronal axons of the mesolimbic dopamine system, a key component of the reward circuitry, generally have high thresholds for stimulation. However, these thresholds can be increased or decreased by drug administration, influencing sensitivity to intracranial stimulation and ICSS behavior.

Monoaminergic drugs

The effects of drugs that alter neurotransmission of dopamine, norepinephrine, and serotonin have been studied extensively in relation to BSR. Neurochemical studies have shown that BSR results in the release of dopamine within the nucleus accumbens. This effect is generally potentiated following administration of drugs that themselves increase the amount of extracellular dopamine in the nucleus accumbens, such as cocaine, which inhibits re-uptake of dopamine to the intracellular space by blocking its transporter. Conversely, these levels are decreased and the rewarding properties of BSR are blocked following administration of drugs that antagonize dopamine receptors or reduce the amount of extracellular dopamine, by promoting either degradation or re-uptake of the neurotransmitter. While dopamine is generally considered to be the main neurotransmitter implicated in the reward system, it is often not the only neurotransmitter affected by addictive, monoaminergic drugs. Importantly, the circuitry involved in BSR is multi-synaptic and not exclusively dopaminergic. This introduces potential for modulation by other neurotransmitter systems at different stages in the transmission of the reward signal.

Additionally, drugs that affect monoamine transmission vary in their selectivity. For example, drugs with high selectivity for increasing dopamine and norepinephrine transmission relative to serotonin transmission tend to have highly addictive properties. Amphetamine and cocaine share this selectivity profile, and administration of these drugs generally results in a left-shift in M50 and θ_0, indicating sensitization of the reward circuitry and high abuse potential. This characteristic leftward-shift of M50 in response to a moderate dose of cocaine is illustrated in a hypothetical data set in Figure 1. Note that the maximum response rate is reached at a lower frequency than it is at baseline, and there is a significant response rate at the lowest frequency, which did not produce responding under baseline conditions. Even following chronic administration of methamphetamine or cocaine, there is little to no tolerance to ICSS facilitation. After chronic treatment is stopped, however, there is a withdrawal-induced depression of ICSS responding, which can be reversed by re-administering the drug.

Drugs with more balanced selectivity for dopamine/norepinephrine and serotonin transmission, such as 3,4-methylenedioxy-methamphetamine (MDMA), tend to be less addictive, and they have mixed effects on ICSS responding depending on dose and stimulation frequency. Serotonin selective drugs, however, tend to result in either a lack of ICSS potentiation or depression of ICSS responding (a right-shift in frequency-rate curve), and these drugs are generally considered to be less addictive. One such example is fenfluramine, which was previously marketed as an appetite suppressant. Dopamine antagonists generally result in the depression of ICSS responding and a rightward-shift in the frequency-rate curve. This suggests decreased BSR and possibly increased aversive properties of the stimulation. Following chronic treatment with a dopamine antagonist, there is withdrawal-induced facilitation of ICSS, the opposite effect of what is observed following chronic treatment with stimulants.

Opioids

Drugs that act on the opioid system generally vary in selectivity for the mu (μ), delta (δ), and kappa (κ) opioid receptors. Their addictive properties are highly dependent on these selectivities. Generally speaking, high potency mu-opioid receptor (MOR) agonists have high abuse potential, while kappa-opioid receptor (KOR) agonists generally produce a dysphoric state. Morphine, a MOR agonist, was one of the earliest studied drugs at the advent of ICSS and BSR. High potency MOR agonists like morphine have a somewhat varied effect on ICSS responding despite having high abuse potential, resulting in both potentiation and depression. The effect these drugs have on ICSS responding has been found to be highly dependent on dose, pretreatment time, and previous opioid exposure. Various studies on the effect of MOR-selective drugs including morphine, heroin, fentanyl, methadone, and hydrocodone have found mixed effects on ICSS responding. Low doses of these drugs have been found to elicit weak facilitation of ICSS, while high doses result in a biphasic ICSS profile, consisting of a higher threshold for ICSS at lower frequencies followed by ICSS potentiation at higher frequencies. Upon chronic administration of high-potency MOR agonists at low doses, there is no tolerance to ICSS facilitation.

Opioid receptor antagonists, such as naloxone, can reverse the effects of both opioid receptor agonists on ICSS responding and the potentiating effects of psychostimulants like methamphetamine. Naloxone, which is a competitive antagonist of all opioid receptor sub-types, does not influence ICSS responding when administered on its own. KOR agonism, typically associated with dysphoric states, more consistently results in a depression of ICSS responding. The KOR agonist salvinorin-A, for example, causes an overall decrease in ICSS response rates at lower stimulation frequencies. Repeated administration does not produce tolerance to ICSS depression. The effects of delta opioid receptor (DOR) agonists/antagonists on ICSS are less clear. One DOR agonist, SNC80, has been found to cause ICSS depression, but there is counter-evidence suggesting some delta agonists might have weak ICSS facilitation properties.

Cholinergic drugs

Cholinergic drugs have been less extensively studied than monoamines and opioids. The most commonly studied cholinergic drug is nicotine, the highly-addictive, psychoactive substance in cigarettes. Nicotine is an agonist of nicotinic acetylcholine receptors (nAchRs), which are ligand-gated ion channels. The addictive properties of nicotine have been found to be associated with agonism specifically of the α4β2 sub-type of nicotinic receptors. Many studies have confirmed that low doses of nicotine result in ICSS facilitation, while higher doses result in ICSS depression. Chronic treatment with nicotine does not result in tolerance to ICSS facilitation at low doses, but does result in tolerance to the depressive effects of high doses. Withdrawal-induced depression of ICSS facilitation at low doses is also observed, as in MOR agonists and monoamine stimulants. The effects of nicotine treatment on ICSS response thresholds and maximum response rates are not as significant as they are in the case of many addictive MOR agonists and monoamine stimulants.

GABAergic drugs

GABAergic transmission is inhibitory, and the two main receptors for GABA are GABA_A and GABA_B. Drugs that act on GABA_A receptors, which are ligand-gated ion channels, are more widely studied, as they generally produce more robust effects on sedation and anxiety, and they are commonly prescribed for therapeutic uses. Ethanol influences GABA receptor activity, and has been found to moderately facilitate ICSS, despite older publications suggesting these findings are inconsistent. Alcohol has several other pharmacological targets other than GABAergic vectors alone, and its metabolite acetaldehyde can activate the mesolimbic dopamine system, particularly at lower levels of drinking.

GABA_B receptor agonists and positive allosteric modulators have been found to result in ICSS depression and have been found to inhibit the reinforcing effects of several drugs, including cocaine, methamphetamine, and nicotine, reversing the ICSS facilitation these drugs typically cause.

Clinical and pre-clinical evidence

Mechanisms of BSR offer a tool that provides insight into the way the brain governs behavior through motivation and reinforcement, especially in regard to addictive and compulsive behaviors. ICSS studies of BSR have proven to be a robust measure of reward sensitivity, and have potential to help assess the abuse liability of various future therapeutics. Additionally, ICSS studies have potential to be used to gauge how reward sensitivity is affected by genetic factors associated with addictive disorders. Drugs found to prevent ICSS facilitation have potential to be developed and therapeutically implemented to reduce the risk of addictive disorders in a clinical setting.