Efficient MCMC with Caching

This post is part of a running series on Bayesian MCMC tutorials. For updates, follow @StableMarkets. Metropolis Review Metropolis-Hastings is an MCMC algorithm for drawing samples from a distribution known up to a constant of proportionality, $latex p(\theta | y) \propto p(y|\theta)p(\theta)$. Very briefly, the algorithm works by starting with some initial draw $latex \theta^{(0)}$ then running … Continue reading Efficient MCMC with Caching

Interactive Tutorial on Dirichlet Processes Using R Shiny

For More posts on Bayesian models, Bayesian Nonparametrics, and causal inference follow me on twitter @stablemarkets. My advisor and his collaborator are teaching a short course on Bayesian Nonparametric Methods for Causal Inference at JSM next week. As part of the short course, I made an interactive tutorial on Dirichlet Processes using R Shiny. All … Continue reading Interactive Tutorial on Dirichlet Processes Using R Shiny

Bayesian Inference with Backfitting MCMC

Previous posts in this series on MCMC samplers for Bayesian inference (in order of publication): Bayesian Simple Linear Regression with Gibbs Sampling in R Blocked Gibbs Sampling in R for Bayesian Multiple Linear Regression Metropolis-in-Gibbs Sampling and Runtime Analysis with Profviz Speeding up Metropolis-Hastings with Rcpp All code for this (and previous) posts are in … Continue reading Bayesian Inference with Backfitting MCMC

Speeding up Metropolis-Hastings with Rcpp

Previous posts in this series on MCMC samplers for Bayesian inference (in order of publication): Bayesian Simple Linear Regression with Gibbs Sampling in R Blocked Gibbs Sampling in R for Bayesian Multiple Linear Regression Metropolis-in-Gibbs Sampling and Runtime Analysis with Profviz The code for all of these posts can be found in my BayesianTutorials GitHub … Continue reading Speeding up Metropolis-Hastings with Rcpp

Metropolis-in-Gibbs Sampling and Runtime Analysis with Profviz

First off, here are the previous posts in my Bayesian sampling series: Bayesian Simple Linear Regression with Gibbs Sampling in R Blocked Gibbs Sampling in R for Bayesian Multiple Linear Regression In the first post, I illustrated Gibbs Sampling - an algorithm for getting draws from a posterior when conditional posteriors are known. In the … Continue reading Metropolis-in-Gibbs Sampling and Runtime Analysis with Profviz

Blocked Gibbs Sampling in R for Bayesian Multiple Linear Regression

In a previous post, I derived and coded a Gibbs sampler in R for estimating a simple linear regression. In this post, I will do the same for multivariate linear regression. I will derive the conditional posterior distributions necessary for the blocked Gibbs sampler. I will then code the sampler and test it using simulated … Continue reading Blocked Gibbs Sampling in R for Bayesian Multiple Linear Regression

Bayesian Simple Linear Regression with Gibbs Sampling in R

Many introductions to Bayesian analysis use relatively simple didactic examples (e.g. making inference about the probability of success given bernoulli data). While this makes for a good introduction to Bayesian principles, the extension of these principles to regression is not straight-forward. This post will sketch out how these principles extend to simple linear regression. Along … Continue reading Bayesian Simple Linear Regression with Gibbs Sampling in R

Fixed Effects, Random Effects, and First Differencing

I came across a stackoverflow post the other day touching on first differencing and decided to write a quick review of the topic as well as related random effects and fixed effects methods. In the end we'll see that random effects, fixed effects, and first differencing are primarily used to handle unobserved heterogeneity within a … Continue reading Fixed Effects, Random Effects, and First Differencing

Exploring P-values with Simulations in R

The recent flare-up in discussions on p-values inspired me to conduct a brief simulation study. In particularly, I wanted to illustrate just how p-values vary with different effect and sample sizes. Here are the details of the simulation. I simulated $latex n $ draws of my independent variable $latex X $: $latex X_n \sim N(100, 400)$ where $latex … Continue reading Exploring P-values with Simulations in R

Stop and Frisk: Spatial Analysis of Racial Differences

In my last post, I compiled and cleaned publicly available data on over 4.5 million stops over the past 11 years. I also presented preliminary summary statistics showing that blacks had been consistently stopped 3-6 times more than whites over the last decade in NYC. Since the last post, I managed to clean and reformat the … Continue reading Stop and Frisk: Spatial Analysis of Racial Differences