Metropolis-in-Gibbs Sampling and Runtime Analysis with Profviz

First off, here are the previous posts in my Bayesian sampling series: Bayesian Simple Linear Regression with Gibbs Sampling in R Blocked Gibbs Sampling in R for Bayesian Multiple Linear Regression In the first post, I illustrated Gibbs Sampling - an algorithm for getting draws from a posterior when conditional posteriors are known. In the … Continue reading Metropolis-in-Gibbs Sampling and Runtime Analysis with Profviz

Advertisements

Blocked Gibbs Sampling in R for Bayesian Multiple Linear Regression

In a previous post, I derived and coded a Gibbs sampler in R for estimating a simple linear regression. In this post, I will do the same for multivariate linear regression. I will derive the conditional posterior distributions necessary for the blocked Gibbs sampler. I will then code the sampler and test it using simulated … Continue reading Blocked Gibbs Sampling in R for Bayesian Multiple Linear Regression

Bayesian Simple Linear Regression with Gibbs Sampling in R

Many introductions to Bayesian analysis use relatively simple didactic examples (e.g. making inference about the probability of success given bernoulli data). While this makes for a good introduction to Bayesian principles, the extension of these principles to regression is not straight-forward. This post will sketch out how these principles extend to simple linear regression. Along … Continue reading Bayesian Simple Linear Regression with Gibbs Sampling in R

Fixed Effects, Random Effects, and First Differencing

I came across a stackoverflow post the other day touching on first differencing and decided to write a quick review of the topic as well as related random effects and fixed effects methods. In the end we'll see that random effects, fixed effects, and first differencing are primarily used to handle unobserved heterogeneity within a … Continue reading Fixed Effects, Random Effects, and First Differencing

Exploring P-values with Simulations in R

The recent flare-up in discussions on p-values inspired me to conduct a brief simulation study. In particularly, I wanted to illustrate just how p-values vary with different effect and sample sizes. Here are the details of the simulation. I simulated $latex n $ draws of my independent variable $latex X $: $latex X_n \sim N(100, 400)$ where $latex … Continue reading Exploring P-values with Simulations in R

Simulating Endogeneity

Introduction The topic in this post is endogeneity, which can severely bias regression estimates. I will specifically simulate endogeneity caused by an omitted variable. In future posts in this series, I'll simulate other specification issues such as heteroskedasticity, multicollinearity, and collider bias. The Data-Generating Process Consider the data-generating process (DGP) of some outcome variable $latex Y $: … Continue reading Simulating Endogeneity

Iterative OLS Regression Using Gauss-Seidel

I just finished covering a few numerical techniques for solving systems of equations, which can be applied to find best-fit lines through a give set of data points. The four points $latex \{(0,0), (1,3), (2,3), (5,6)\}$ are arranged into an inconsistent system of four equations and two unknowns: $latex b+a(0) = 0 \\ b+a(1) = 3 \\ b+a(2) … Continue reading Iterative OLS Regression Using Gauss-Seidel