Cover

Bayesian Nonparametrics

Title

Copyright

Contents

Contributors

An invitation to Bayesian nonparametrics

Bayesian nonparametrics

What is it all about?

Who needs it?

Why now?

The aims, purposes and contents of this book

A background event

What does this book do?

How do alternative models relate to each other?

How to teach from this book

A brief history of Bayesian nonparametrics

From the start to the present

Applications

Where does this book fit in the broader picture?

Further topics

Computation and software\sindexsoftware packages

Challenges and future developments

References

1 Bayesian nonparametric methods: motivation and ideas

1.1 Introduction

1.2 Bayesian choices

1.3 Decision theory

1.4 Asymptotics

1.5 General posterior inference

1.6 Discussion

References

2 The Dirichlet process, related priors and posterior asymptotics

2.1 Introduction

2.2 The Dirichlet process

2.2.1 Motivation

2.2.2 Construction of the Dirichlet process\sindexDirichlet process!construction

Naive construction

Construction using a countable generator

Construction by normalization

2.2.3 Properties

Moments and marginal distribution

Linear functionals

Conjugacy

Posterior mean

Limits of the posterior

Lack of smoothness

Negative correlation

Discreteness

Support

Self-similarity

Limit types

Dirichlet samples and ties

Sethuraman stick-breaking representation

Mutual singularity

Tail of a Dirichlet process

2.3 Priors related to the Dirichlet process

2.3.1 Mixtures of Dirichlet processes

2.3.2 Dirichlet process mixtures

2.3.3 Hierarchical Dirichlet processes

2.3.4 Invariant and conditioned Dirichlet processes

2.4 Posterior consistency

2.4.1 Motivation and implications

2.4.2 Doob's theorem

2.4.3 Instances of inconsistency

2.4.4 Approaches to consistency

2.4.5 Schwartz's theory

Kullback–Leibler property

Bounding the numerator

Uniformly consistent tests

Entropy and sieves

2.4.6 Density estimation

Dirichlet mixtures

Gaussian processes

Pólya tree processes

2.4.7 Semiparametric applications

2.4.8 Non-i.i.d. observations

2.4.9 Sieve-free approaches

Martingale method

Power-posterior distribution

2.5 Convergence rates of posterior distributions

2.5.1 Motivation, description and consequences

2.5.2 General theory

Prior concentration rate

Entropy and tests

Sieves

2.5.3 Applications

Optimal rates using brackets

Finite-dimensional models

Log-spline priors

Dirichlet mixtures

Gaussian processes

2.5.4 Misspecified models

2.5.5 Non-i.i.d. extensions

2.6 Adaptation and model selection

2.6.1 Motivation and description

2.6.2 Infinite-dimensional normal models

2.6.3 General theory of Bayesian adaptation

2.6.4 Density estimation using splines

2.6.5 Bayes factor consistency

2.7 Bernshteǐn–von Mises theorems

2.7.1 Parametric Bernshteǐn–von Mises theorems

2.7.2 Nonparametric Bernshteǐn–von Mises theorems

2.7.3 Semiparametric Bernshteǐn–von Mises theorems

2.7.4 Nonexistence of Bernshteǐn–von Mises theorems

2.8 Concluding remarks

References

3 Models beyond the Dirichlet process

3.1 Introduction

3.1.1 Exchangeability assumption

3.1.2 A concise account of completely random measures

3.2 Models for survival analysis

3.2.1 Neutral-to-the-right priors

3.2.2 Priors for cumulative hazards: the beta process

3.2.3 Priors for hazard rates

3.3 General classes of discrete nonparametric priors

3.3.1 Normalized random measures with independent increments

3.3.2 Exchangeable partition probability function

3.3.3 Poisson–Kingman models and Gibbs-type priors

3.3.4 Species sampling models

3.4 Models for density estimation

3.4.1 Mixture models

3.4.2 Pólya trees

3.5 Random means

3.6 Concluding remarks

References

4 Further models and applications

4.1 Beta processes for survival and event history models

4.1.1 Construction and interpretation

4.1.2 Transitions and Markov processes

4.1.3 Hazard regression models

4.1.4 Semiparametric competing risks models

4.2 Quantile inference

4.3 Shape analysis

4.4 Time series with nonparametric correlation function

4.5 Concluding remarks

4.5.1 Bernshteǐn–von Mises theorems

4.5.2 Mixtures of beta processes

4.5.3 Bayesian kriging

4.5.4 From nonparametric Bayes to parametric survival models

References

5 Hierarchical Bayesian nonparametric models with applications

5.1 Introduction

5.2 Hierarchical Dirichlet processes

5.2.1 Stick-breaking construction

5.2.2 Chinese restaurant franchise

5.2.3 Posterior structure of the HDP

5.2.4 Applications of the HDP

Information retrieval

Multipopulation haplotype phasing

Topic modeling

5.3 Hidden Markov models with infinite state spaces

5.3.1 Applications of the HDP-HMM

Speaker diarization

Word segmentation

Trees and grammars

5.4 Hierarchical Pitman–Yor processes

5.4.1 Pitman–Yor processes

5.4.2 Hierarchical Pitman–Yor processes

5.4.3 Applications of the hierarchical Pitman–Yor process

5.5 The beta process and the Indian buffet process

5.5.1 The beta process and the Bernoulli process

5.5.2 The Indian buffet process

5.5.3 Stick-breaking constructions

5.5.4 Hierarchical beta processes

5.5.5 Applications of the beta process

Sparse latent variable models

Relational models

5.6 Semiparametric models

5.6.1 Hierarchical DPs with random effects

5.6.2 Analysis of densities and transformed DPs

5.7 Inference for hierarchical Bayesian nonparametric models

5.7.1 Inference for hierarchical Dirichlet processes

Chinese restaurant franchise sampler

Posterior representation sampler

5.7.2 Inference for HDP hidden Markov models

5.7.3 Inference for beta processes

5.7.4 Inference for hierarchical beta processes

5.8 Discussion

References

6 Computational issues arising in Bayesian nonparametric hierarchical models

6.1 Introduction

6.2 Construction of finite-dimensional measures on observables

6.3 Recent advances in computation for Dirichlet process mixture models

References

7 Nonparametric Bayes applications to biostatistics

7.1 Introduction

7.2 Hierarchical modeling with Dirichlet process priors

7.2.1 Illustration for simple repeated measurement models

7.2.2 Posterior computation

7.2.3 General random effects models

7.2.4 Latent factor regression models

7.3 Nonparametric Bayes functional data analysis

7.3.1 Background

7.3.2 Basis functions and clustering

7.3.3 Functional Dirichlet process

7.3.4 Kernel-based approaches

7.3.5 Joint modeling

7.4 Local borrowing of information and clustering

7.5 Borrowing information across studies and centers

7.6 Flexible modeling of conditional distributions

7.6.1 Motivation

7.6.2 Dependent Dirichlet processes

7.6.3 Kernel-based approaches

7.6.4 Conditional distribution modeling through DPMs

7.6.5 Reproductive epidemiology application

7.7 Bioinformatics

7.7.1 Modeling of differential gene expression

7.7.2 Analyzing polymorphisms and haplotypes

7.7.3 New species discovery

7.8 Nonparametric hypothesis testing

7.9 Discussion

References

8 More nonparametric Bayesian models for biostatistics

8.1 Introduction

8.2 Random partitions

8.3 Pólya trees

8.4 More DDP models

8.4.1 The ANOVA DDP

8.4.2 Classification with DDP models

8.5 Other data formats

8.6 An R package for nonparametric Bayesian inference

8.7 Discussion

References

Author index

Subject index