Stochastic Methods

This course covers foundational and advanced topics in stochastic processes, with a focus on both theory and practical applications. Students learn to model randomness in systems and analyze probabilistic phenomena through various methods.

Course Overview

The course explores a wide array of stochastic models and techniques, including:

• Random Variables and Distributions: Understanding basic probability concepts like random variables, expectation, variance, and moments, along with specific distributions such as binomial, Poisson, and normal distributions.

• Markov Chains: Studying stochastic processes where the future state depends only on the current state. Topics include transition matrices, Chapman-Kolmogorov equations, and limit distributions.

• Poisson Processes: Delving into counting processes, particularly the Poisson process and its variants such as homogeneous and non-homogeneous Poisson processes.

• Random Networks: Introducing random graphs, branching processes, and Markov Decision Processes (MDP), including real-world applications in network analysis and decision-making.

• Monte Carlo Simulation: Learning about Monte Carlo methods, random number generators, and simulations of both discrete and continuous random variables.

• Stochastic Optimization: Covering topics such as evolutionary algorithms, stochastic gradient descent, and the expectation-maximization (EM) algorithm.

Key Learning Objectives

1. Master Stochastic Concepts: Grasp core principles of stochastic methods, including understanding and applying random variables, distributions, and stochastic processes.

2. Analyze Markov Chains: Develop the ability to model and analyze Markov Chains, exploring their long-term behavior through limit theorems and transition matrices.

3. Apply Stochastic Simulation: Use Monte Carlo methods and stochastic optimization to solve real-world problems, focusing on both discrete and continuous scenarios.

4. Explore Network Models: Understand how random networks function, with a focus on network structures and stochastic models such as branching processes.

5. Optimize Through Stochastic Algorithms: Learn to apply algorithms like stochastic gradient descent and MCMC methods in optimization problems.

Example Topics Covered:

• Expectation and Variance of Random Variables
• Continuous-Time Markov Chains and Birth-Death Processes
• Renewal and Queuing Theories
• Poisson Process Applications in Time and Space
• Simulation Techniques: Monte Carlo Methods and Random Number Generation
• Optimization Techniques: Stochastic Gradient Descent and Markov Chain Monte Carlo (MCMC)

For further reading, please Download the Stochastic Methods lecture notes.