Stochastic Methods

This course covers foundational and advanced topics in stochastic processes, with a focus on both theory and practical applications. Students will learn to model randomness in systems and analyze probabilistic phenomena through various methods.

Course Overview

The course explores a wide array of stochastic models and techniques, including:

• Random Variables and Distributions: Understanding basic probability concepts like random variables, expectation, variance, and moments, along with specific distributions such as binomial, Poisson, and normal distributions.

• Markov Chains: Studying stochastic processes where the future state depends only on the current state. Topics include transition matrices, Chapman-Kolmogorov equations, and limit distributions.

• Poisson Processes: Delving into counting processes, particularly the Poisson process and its variants such as homogeneous and non-homogeneous Poisson processes.

• Random Networks: Introducing random graphs, branching processes, and Markov Decision Processes (MDP), including real-world applications in network analysis and decision-making.

• Monte Carlo Simulation: Learning about Monte Carlo methods, random number generators, and simulations of both discrete and continuous random variables.

• Stochastic Optimization: Covering topics such as evolutionary algorithms, stochastic gradient descent, and the expectation-maximization (EM) algorithm.

For further reading, please (Download the Stochastic Methods lecture notes ).