MonteCarlo-Statistical-Methods

A visual and practical guide to Monte Carlo techniques, focused on understanding rather than memorizing.
Each method is implemented from scratch and explored through intuitive diagrams, animations, and diagnostic tools.
These foundations support real applications in robotics, dynamics, and reinforcement learning, from Bayesian filters to sampling-based planning and uncertainty-aware control.

---

Table of Contents

• Overview
• Sampling Visualizations
• Importance Sampling
• Control Variates
• Brownian Motion
• Markov Chain Monte Carlo
• Notebook Gallery
• PDF Chapter Library
• Project Structure
• About Me

---

Overview

This repository builds Monte Carlo ideas from first principles using:

• Random variable generation: inverse transform, accept–reject, importance sampling
• Monte Carlo estimation: WLLN, SLLN, CLT
• Variance reduction: control variates, Brownian motion scaling
• Markov Chain Monte Carlo: MH, AM, DR, DRAM
• Full diagnostics: burn-in, mixing, autocorrelation, integrated autocorrelation, ESS

Each topic is supported by clean visualizations and self-contained Jupyter notebooks.

---

Sampling Visualizations

   

Inverse Transform sampling (left) and Accept–Reject sampling (right).

---

Importance Sampling

   

Proposal vs likelihood alignment and resulting posterior for Importance Sampling.

---

Control Variates

   

Variance comparison (left) and correlation structure enabling control variates (right).

---

Brownian Motion

   

Random-walk scaling to Brownian motion and simulated sample paths.

---

Markov Chain Monte Carlo

The repository implements four MCMC algorithms on both 2D Gaussian and banana-shaped targets:

• Metropolis–Hastings (MH)
• Adaptive Metropolis (AM)
• Delayed Rejection (DR)
• Delayed Rejection Adaptive Metropolis (DRAM)

All diagnostics: burn-in, mixing, autocorrelation, integrated autocorrelation, and ESS, are derived in the MH diagnostics notebook and reused across the remaining algorithms.

Below is a DRAM example on the banana-shaped target.

Target and Laplace Initialization

DRAM Sample Exploration

Mixing Behavior

Autocorrelation Diagnostics

---

Notebook Gallery

Chapter 2: Sampling

• General Transformations
• Accept–Reject Sampling
• Exponential RVs
• Gamma RVs

Chapter 3: Importance Sampling

• Cauchy Tail Motivation
• Rare Event Estimation
• Self-Normalized IS

Chapter 4: Variance Reduction

• Control Variate Example

Chapter 6: Stochastic Processes

• Brownian Motion

Chapter 5: MCMC

• MH Diagnostics
• Adaptive Metropolis — Gaussian
• Adaptive Metropolis — Banana
• Delayed Rejection — Gaussian
• Delayed Rejection — Banana
• DRAM — Gaussian
• DRAM — Banana

---

PDF Chapter Library

Full write-ups:

• reports/ch02_general_transforms.pdf
• reports/ch02_accept_reject.pdf
• reports/ch03_01_ImportantSampling_Motivation_weights.pdf
• reports/ch03_02_ImportanceSampling_MC_vs_IS_Variance_Comparison.pdf
• reports/ch03_03_ImportanceSampling_Rare_event_Estimation.pdf
• reports/ch03_04_SelfNormalized_ImportantSampling.pdf
• reports/ch04_01_ControlVariate_Foundations-and-Intution.pdf
• reports/ch04_02_Brownian_Motion.pdf
• reports/ch04_03_ControlVariate_example1.pdf
• reports/ch05_01_MarkovChain_Intro.pdf
• reports/ch05_02_Irreducibility.pdf
• reports/ch05_07_Metropolis-Hastings.pdf

---

Project Structure

MonteCarlo-Statistical-Methods/
│
├── animations/
├── images/
│   ├── sampling/
│   ├── importance_sampling/
│   ├── variance_reduction/
│   ├── stochastic_processes/
│   ├── mcmc/
│   └── exponential/
├── notebooks/
│   ├── ch02_sampling/
│   ├── ch03_importance_sampling/
│   ├── ch04_variance_reduction/
│   ├── ch05_mcmc/
│   └── ch06_stochastic_processes/
├── reports/
├── src/
├── README.md
├── index.md
└── pyproject.toml

---

About Me

I study and implement methods in optimization, control, robotics, Bayesian inference, and probabilistic reasoning.

• GitHub: https://github.com/SaiSampathKedari
• LinkedIn: https://linkedin.com/in/sai-sampath-kedari
• X: https://x.com/SSampathKedari
• Email: sampath@umich.edu