Front Cover

DECISION NEUROSCIENCE

DECISION NEUROSCIENCE:AN INTEGRATIVE PERSPECTIVE

Copyright

Contents

List of Contributors

Preface

STRUCTURE OF THE BOOK

I - ANIMAL STUDIES ON REWARDS, PUNISHMENTS, AND DECISION-MAKING

1 - Anatomy and Connectivity of the Reward Circuit

INTRODUCTION

PREFRONTAL CORTEX

THE VENTRAL STRIATUM

Special Features of the Ventral Striatum

Connections of the Ventral Striatum (See Fig. 1.3)

Cortical Projections to the Ventral Striatum

The Amygdala and Hippocampal Projections to the Ventral Striatum

Thalamic Projections to the Ventral Striatum

Efferent Projections From the Ventral Striatum (See Fig. 1.4)

VENTRAL PALLIDUM (FIG. 1.4)

THE MIDBRAIN DOPAMINE NEURONS

Afferent Projections (Fig. 1.5B)

Efferent Projections (Fig. 1.5B)

The Striatonigrostriatal Network

COMPLETING THE CORTICAL–BASAL GANGLIA REWARD CIRCUIT

Subthalamic Nucleus, the Hyperdirect Cortical Pathway

The Lateral Habenula, a Negative-Reward Signal

The Thalamus, the Final Link Back to Cortex

The Lateral Habenula, Pedunculopontine Tegmental Nucleus, and the Raphe Serotonergic Systems

Complex Network Features of the Reward Circuit

References

2 - Electrophysiological Correlates of Reward Processing in Dopamine Neurons

INTRODUCTION

BASICS OF DOPAMINE REWARD-PREDICTION ERROR CODING

Reward-Prediction Error

Response Components

Reward Learning

SUBJECTIVE VALUE AND FORMAL ECONOMIC UTILITY CODING

Subjective Value

Subjective Value Derived From Risk

Utility

DOPAMINE AND DECISION-MAKING

Updating of Decision Variables

Direct Dopamine Influences on Choices

Acknowledgments

References

3 - Appetitive and Aversive Systems in the Amygdala

INTRODUCTION

Valence in the Amygdala

Updating the Valence Representation in the Amygdala: Learning, Extinction, and Cognitive Regulation

The Influence of Valenced Representations in the Amygdala on Cognitive Processes

CONCLUSION

References

4 - Ventral Striatopallidal Pathways Involved in Appetitive and Aversive Motivational Processes

INTRODUCTION

THE CORTICOBASAL GANGLIA FUNCTIONAL CIRCUITS: THE RELATION BETWEEN THE CORTEX AND THE BASAL GANGLIA

THE DIRECT AND INDIRECT PATHWAYS: FROM INHIBITION OF COMPETITIVE MOVEMENT TO AVERSIVE BEHAVIORS

SINGLE-UNIT RECORDING IN AWAKE ANIMALS TO INVESTIGATE THE NEURAL BASES

TASKS TO INVESTIGATE APPETITIVE APPROACH BEHAVIOR AND POSITIVE MOTIVATION

VENTRAL STRIATUM AND VENTRAL PALLIDUM ARE ALSO INVOLVED IN AVERSIVE BEHAVIORS: FIRST EVIDENCE OF LOCAL INHIBITORY DYSFUNCTION

THE VENTRAL STRIATUM AND THE VENTRAL PALLIDUM ENCODE AVERSIVE FUTURE EVENTS FOR PREPARATION OF AVOIDANCE AND FOR CONTROLLING ANXIETY LEVEL

ABNORMAL AVERSIVE PROCESSING COULD BIAS DECISION-MAKING TOWARD PATHOLOGICAL BEHAVIORS

CONCLUSION

Acknowledgments

References

5 - Reward and Decision Encoding in Basal Ganglia: Insights From Optogenetics and Viral Tracing Studies in Rodents

DOPAMINE

Physiology

Function

Circuitry

STRIATUM

Initiating Movement, Habits, and Motor Skills

Value Representations and Adaptive Decision-Making

Circuits

CONCLUSIONS

References

6 - The Learning and Motivational Processes Controlling Goal-Directed Action and Their Neural Bases

LEARNING “WHAT LEADS TO WHAT”: THE NEURAL BASES OF ACTION–OUTCOME LEARNING

HOW REWARDS ARE FORMED: NEURAL BASES OF INCENTIVE LEARNING

DECIDING WHAT TO DO: THE NEURAL BASES OF OUTCOME RETRIEVAL AND CHOICE

HOW THESE CIRCUITS INTERACT: CONVERGENCE ON THE DORSOMEDIAL STRIATUM

CONCLUSION

Acknowledgments

References

7 - Impulsivity, Risky Choice, and Impulse Control Disorders: Animal Models

DELAYED AND PROBABILISTIC DISCOUNTING: “IMPULSIVE CHOICE” IN DECISION-MAKING

PREMATURE RESPONDING IN THE 5CSRTT

STOP-SIGNAL REACTION TIME

NEURAL BASIS OF IMPULSIVITY

NEURAL SUBSTRATES OF WAITING IMPULSIVITY

5CSRTT Premature Responding

Temporal Discounting of Reward

Nucleus Accumbens

Amygdala and Corticolimbic Influences

NEURAL SUBSTRATES OF PROBABILITY DISCOUNTING OF REWARD: RISKY CHOICE

MOTOR IMPULSIVITY: STOP-SIGNAL INHIBITION

CONCLUSION

Acknowledgments

References

8 - Prefrontal Cortex in Decision-Making: The Perception–Action Cycle

THE PERCEPTION–ACTION CYCLE

INPUTS TO THE PERCEPTION–ACTION CYCLE IN DECISION-MAKING

PREDICTION AND PREPARATION TOWARD DECISION

EXECUTION OF DECISION

FEEDBACK FROM DECISION: CLOSURE OF THE PERCEPTION–ACTION CYCLE

References

II - HUMAN STUDIES ON MOTIVATION, PERCEPTUAL, AND VALUE-BASED DECISION-MAKING

9 - Reward, Value, and Salience

INTRODUCTION

VALUE

Definition and Behavioral Function

Value Signals in the Human Brain

SALIENCE

Definition and Behavioral Function

Confounds in Operational Definitions of Salience

Salience Signals in the Human Brain

CONCLUSIONS

References

10 - Computational Principles of Value Coding in the Brain

INTRODUCTION

VALUE AND CHOICE BEHAVIOR

VALUE CODING IN DECISION CIRCUITS

CONTEXT DEPENDENCE IN BRAIN AND BEHAVIOR

NEURAL COMPUTATIONS UNDERLYING VALUE REPRESENTATION

TEMPORAL DYNAMICS AND CIRCUIT MECHANISMS

CONCLUSIONS

References

11 - Spatiotemporal Characteristics and Modulators of Perceptual Decision-Making in the Human Brain

INTRODUCTION

FACTORS AFFECTING PERCEPTUAL DECISION-MAKING

Prestimulus State

Reward and Punishment

Speed Versus Accuracy Trade-Off

Learning and Training

Confidence

Neuromodulation

CONCLUSION

References

12 - Perceptual Decision-Making: What Do We Know, and What Do We Not Know?

INTRODUCTION

WHAT ARE PERCEPTUAL DECISIONS?

AIMS AND SCOPE OF THIS CHAPTER

DECISION OPTIMALITY

Q1: HOW IS INFORMATION INTEGRATED DURING PERCEPTUAL DECISION-MAKING?

Observers Sample Information Sequentially

However, Information Integration Is Not Lossless

Q1: Summary

Q2: WHAT COMPUTATIONS DO CORTICAL NEURONS PERFORM DURING PERCEPTUAL DECISIONS?

In the Macaque, Lateral Intraparietal Region Firing Rates Grow During Information Integration

In the Rodent, Posterior Parietal Cortex Neurons Show Similar Dynamics

However, Lateral Intraparietal Region and Posterior Parietal Cortex Neurons Exhibit Mixed Selectivity

Q2: Summary

Q3: HOW CAN WE STUDY PERCEPTUAL DECISION-MAKING IN HUMANS?

It Is Unclear How BOLD Signals Encode Decision Variables

EEG Signals Capture Integration-to-Bound Dynamics During Perceptual Decisions

EEG Studies Suggest Discrete Computational Stages for Perceptual Decisions

Q3: Summary

Q4: HOW DO OBSERVERS DECIDE WHEN TO DECIDE?

It Is Unclear How Speed and Accuracy Are Traded Off at the Neural Level

The Optimal Bound Collapses When Sensory Reliability Is Unknown

Decisions May Be Driven by a Time-Varying Urgency Signal

Q4: Summary

Q5: HOW ARE PERCEPTUAL DECISIONS BIASED BY PRIOR BELIEFS?

Probability Cues Can Bias Both the Origin and the Slope of Information Integration

Predictive Coding

Adaptive Gain Control During Perceptual Decision-Making

Q5: Summary

CONCLUSIONS AND FUTURE DIRECTIONS

Acknowledgments

References

13 - Neural Circuit Mechanisms of Value-Based Decision-Making and Reinforcement Learning

INTRODUCTION

REPRESENTATIONS OF REWARD VALUE

LEARNING REWARD VALUES

STOCHASTIC DOPAMINE-DEPENDENT PLASTICITY FOR LEARNING REWARD VALUES

FORAGING WITH PLASTIC SYNAPSES

RANDOM CHOICE AND COMPETITIVE GAMES

PROBABILISTIC INFERENCE WITH STOCHASTIC SYNAPSES

CONCLUDING REMARKS

Acknowledgments

References

III - SOCIAL DECISION NEUROSCIENCE

14 - Social Decision-Making in Nonhuman Primates

INTRODUCTION

BEHAVIORAL STUDIES ON SOCIAL DECISION-MAKING

NEURONAL CORRELATES OF DECISION-MAKING IN A SOCIAL CONTEXT

CONCLUSIONS AND PERSPECTIVES

References

15 - Organization of the Social Brain in Macaques and Humans

INTRODUCTION

MEDIAL PREFRONTAL CORTEX

SUPERIOR TEMPORAL SULCUS AND TEMPORAL PARIETAL JUNCTION

A SOCIAL BRAIN NETWORK

SUMMARY AND PERSPECTIVES

References

16 - The Neural Bases of Social Influence on Valuation and Behavior

INTRODUCTION

THE EFFECT OF MERE PRESENCE OF OTHERS ON PROSOCIAL BEHAVIOR

The Neural Bases of Social Reward Processing

The Observer Effect in Autism

THE EFFECT OF OTHERS' OPINIONS ON VALUATION

The Neural Bases of Social Conformity

Summary and Future Directions

CONCLUDING REMARKS

References

17 - Social Dominance Representations in the Human Brain

INTRODUCTION

Social Hierarchies in Health and Well-Being

Social Hierarchies as a Major Evolutionary Pressure and Pivotal Feature of Societies

LEARNING SOCIAL DOMINANCE HIERARCHIES

Reinforcement Learning Approaches to Social Cognition

The Emergence of Dominance and Subordination by Reinforcement Learning

INTERINDIVIDUAL DIFFERENCES AND SOCIAL DOMINANCE

Intercultural Differences in the Appraisal of Social Dominance

Interindividual Differences Resulting From Social Status and Personality

NEUROCHEMICAL APPROACHES TO SOCIAL DOMINANCE AND SUBORDINATION

Stress Asymmetries Paralleling Social Hierarchy Rank Have Adverse Consequences on Adrenocortical, Reproductive, and Neural ...

Dopamine, Serotonin, and Social Hierarchies in Rodents and Nonhuman Primates

CONCLUSION

Acknowledgments

References

18 - Reinforcement Learning and Strategic Reasoning During Social Decision-Making

INTRODUCTION

MODEL-FREE VERSUS MODEL-BASED REINFORCEMENT LEARNING

NEURAL CORRELATES OF MODEL-FREE REINFORCEMENT LEARNING DURING SOCIAL DECISION-MAKING

NEURAL CORRELATES OF HYBRID REINFORCEMENT LEARNING DURING SOCIAL DECISION-MAKING

ARBITRATION AND SWITCHING BETWEEN LEARNING ALGORITHMS

CONCLUSIONS

References

19 - Neural Control of Social Decisions: Causal Evidence From Brain Stimulation Studies

INTRODUCTION

NONINVASIVE BRAIN STIMULATION METHODS USED FOR STUDYING SOCIAL DECISIONS

BRAIN STIMULATION STUDIES OF SOCIAL EMOTIONS

BRAIN STIMULATION STUDIES OF SOCIAL COGNITION

BRAIN STIMULATION STUDIES OF SOCIAL BEHAVIORAL CONTROL

BRAIN STIMULATION EVIDENCE FOR SOCIAL-SPECIFIC NEURAL ACTIVITY

CONCLUSIONS

References

20 - The Neuroscience of Compassion and Empathy and Their Link to Prosocial Motivation and Behavior

INTRODUCTION

THE “TOOLKIT” OF SOCIAL COGNITION

Mentalizing

Mimicry and Emotional Contagion

THE NEURAL SUBSTRATES OF EMPATHY

THE PSYCHOLOGICAL AND NEURAL BASES OF COMPASSION

A Missing Link Between Empathy and Prosocial Motivation

The Psychological Bases of Compassion

The Neural Bases of Compassion

Compassion and Prosocial Behavior

CONCLUSION

References

IV - HUMAN CLINICAL STUDIES INVOLVING DYSFUNCTIONS OF REWARD AND DECISION-MAKING PROCESSES

21 - Can Models of Reinforcement Learning Help Us to Understand Symptoms of Schizophrenia?

INTRODUCTION

REWARD PROCESSING IN SCHIZOPHRENIA: A HISTORICAL PERSPECTIVE

DOPAMINE, SCHIZOPHRENIA, AND REINFORCEMENT LEARNING

THE POSSIBLE IMPORTANCE OF GLUTAMATE

STUDIES OF REWARD PROCESSING/REINFORCEMENT LEARNING IN PSYCHOSIS: BEHAVIORAL STUDIES

STUDIES OF REWARD PROCESSING/REINFORCEMENT LEARNING IN PSYCHOSIS: NEUROIMAGING STUDIES

CAN AN UNDERSTANDING OF REWARD AND DOPAMINE HELP US TO UNDERSTAND SYMPTOMS OF SCHIZOPHRENIA?

Delusions

Anhedonia

SUMMARY

Acknowledgments

References

22 - The Neuropsychology of Decision-Making: A View From the Frontal Lobes

INTRODUCTION

LESION EVIDENCE IN HUMANS

Lesion Study Designs

DECISION-MAKING AND THE FRONTAL LOBES

COMPONENT PROCESSES OF DECISION-MAKING

Overview

Option Identification

Value Construction

Learning Value

Value and Attention

Decision Strategies

Decisions Over Time

SUMMARY

Acknowledgments

References

23 - Opponent Brain Systems for Reward and Punishment Learning: Causal Evidence From Drug and Lesion Studies in Humans

THE NEURAL CANDIDATES FOR REWARD AND PUNISHMENT LEARNING SYSTEMS

Hypothesis 1: No Opponent System

Hypothesis 2: Dopaminergic Opponent System

Hypothesis 3: Serotoninergic Opponent System

Hypothesis 4: Other Opponent Systems

EVIDENCE FROM DRUG AND LESION STUDIES

CONCLUSIONS, LIMITATIONS, AND PERSPECTIVES

References

24 - Decision-Making and Impulse Control Disorders in Parkinson's Disease

INTRODUCTION

THE ROLE OF DOPAMINERGIC MEDICATIONS AND INDIVIDUAL VULNERABILITY IN PARKINSON'S DISEASE

REINFORCING EFFECTS AND ASSOCIATIVE LEARNING

LEARNING FROM FEEDBACK

RISK AND UNCERTAINTY

IMPULSIVITY

SUMMARY

References

25 - The Subthalamic Nucleus in Impulsivity

INTRODUCTION

ANATOMY, PHYSIOLOGY, AND FUNCTION OF CORTICOBASAL GANGLIA CIRCUITS

THE SUBTHALAMIC NUCLEUS AND DECISION-MAKING: EVIDENCE FROM ANIMAL STUDIES

THE SUBTHALAMIC NUCLEUS AND IMPULSIVITY: EVIDENCE FROM BEHAVIORAL OBSERVATIONS

THE SUBTHALAMIC NUCLEUS AND DECISION-MAKING: EVIDENCE FROM NEUROPSYCHOLOGICAL STUDIES

A MODEL OF THE IMPACT OF THE SUBTHALAMIC NUCLEUS ON DECISION-MAKING

References

26 - Decision-Making in Anxiety and Its Disorders

INTRODUCTION

Representation

Valuation

Action Selection

Outcome Evaluation

Learning

SUMMARY AND CONCLUSIONS

References

27 - Decision-Making in Gambling Disorder: Understanding Behavioral Addictions

INTRODUCTION: GAMBLING AND DISORDERED GAMBLING

LOSS AVERSION

PROBABILITY WEIGHTING

PERCEPTIONS OF RANDOMNESS

ILLUSORY CONTROL

CONCLUSION

Disclosures

References

V - GENETIC AND HORMONAL INFLUENCES ON MOTIVATION AND SOCIAL BEHAVIOR

28 - Decision-Making in Fish: Genetics and Social Behavior

SOCIAL SYSTEM OF THE AFRICAN CICHLID FISH, ASTATOTILAPIA BURTONI

Social System of Astatotilapia burtoni: Consequences of Status

DOMAINS OF ASTATOTILAPIA BURTONI SOCIAL DECISIONS

Can Males Be Deceptive?

Attention Hierarchy Among Male Fish

When Being Observed, Males Change Their Behavior

Transitive Inference by Males

Genomic Consequences of Female Mate Choice

SUMMARY

References

29 - Imaging Genetics in Humans: Major Depressive Disorder and Decision-Making

INTRODUCTION

MAJOR DEPRESSIVE DISORDER AS A DISORDER OF DECISION-MAKING

IMAGING GENETICS OF MAJOR DEPRESSIVE DISORDER

CONCLUSION

References

30 - Time-Dependent Shifts in Neural Systems Supporting Decision-Making Under Stress

INTRODUCTION

LARGE-SCALE NEUROCOGNITIVE SYSTEMS AND SHIFTS IN RESOURCE ALLOCATION

SALIENCE NETWORK AND ACUTE STRESS

Salience Network: Neuroendocrine and Cellular Effects of Acute Stress

Salience Network: Behavioral Effects of Acute Stress

Salience Network: Brain Systems-Level Effects of Acute Stress

EXECUTIVE CONTROL NETWORK AND ACUTE STRESS

Executive Control Network: Behavioral Effects of Acute Stress

Executive Control Network: Brain Systems-Level Effects of Acute Stress

SALIENCE NETWORK AND EXECUTIVE CONTROL NETWORK AND RECOVERY FROM STRESS

SUMMARY AND CONCLUSION

References

31 - Oxytocin's Influence on Social Decision-Making

INTRODUCTION

The Social Brain

Oxytocin's Dual Mode of Release in the Brain

OXYTOCIN AND PERCEPTION OF SOCIAL STIMULI

Sensory Perception

Emotion Perception

OXYTOCIN AND SOCIAL DECISIONS

Trust Games and Cooperative Behaviors

Empathy and Prosociality

Social Norms

OXYTOCIN AND SOCIAL REWARD

Social Reward

Nonsocial Reward

OXYTOCIN, LEARNING, AND MEMORY

PERSPECTIVES

Acknowledgments

References

32 - Appetite as Motivated Choice: Hormonal and Environmental Influences

INTRODUCTION

APPETITIVE BRAIN SYSTEMS PROMOTE FOOD INTAKE

Hippocampus and Amygdala

Insula

Orbitofrontal Cortex

Striatum

Prefrontal Regions

SELF-CONTROL AND LATERAL PREFRONTAL CORTEX: ROLE IN APPETITE REGULATION AND OBESITY

INTERACTION BETWEEN ENERGY BALANCE SIGNALS AND DECISION-MAKING

Insulin

Leptin

Ghrelin

Satiety Hormones: Glucagon-Like Peptide 1 and Peptide YY

CONCLUSION

References

33 - Perspectives

IDENTIFYING FUNDAMENTAL COMPUTATIONAL PRINCIPLES: PRODUCE CONCEPTUAL FOUNDATIONS FOR UNDERSTANDING THE BIOLOGICAL BASIS OF ...

UNDERSTANDING THE FUNCTIONAL ORGANIZATION OF THE PREFRONTAL CORTEX AND THE NATURE OF THE COMPUTATIONS PERFORMED IN VARIOUS SUBREGIONS: VALUE-CODING COMPUTATIONS

DEMONSTRATING CAUSALITY: LINKING BRAIN ACTIVITY TO BEHAVIOR BY DEVELOPING AND APPLYING PRECISE INTERVENTIONAL TOOLS THAT CHANGE NEURAL CIRCUIT DYNAMICS

MAPS AT MULTIPLE SCALES: GENERATE CIRCUIT DIAGRAMS THAT VARY IN RESOLUTION FROM SYNAPSES TO THE WHOLE BRAIN

THE BRAIN IN ACTION: PRODUCE A DYNAMIC PICTURE OF THE FUNCTIONING BRAIN BY DEVELOPING AND APPLYING IMPROVED METHODS FOR LARGE-SCALE MONITORING OF NEURAL ACTIVITY

THE ANALYSIS OF CIRCUITS OF INTERACTING NEURONS

DEVELOP INNOVATIVE TECHNOLOGIES AND SIMULTANEOUS MEASURES TO UNDERSTAND HOW THE BRAIN MAKES DECISIONS

ADVANCING HUMAN DECISION NEUROSCIENCE: UNDERSTANDING NEUROLOGICAL/PSYCHIATRIC DISORDERS AND TREATING BRAIN DISEASES

CONCLUSIONS

Reference

Index

A

B

C

D

E

F

G

H

I

L

M

N

O

P

R

S

T

U

V

W

Back Cover