Stochastic Modeling with Markov Chains

Dr. Thuan Nguyen


This course aims to provide an introduction on Markov chains in discrete time. The main content includes:

  • Markov chains; Transition probabilities; Chapman-Kolmogorov equations;
  • Classification of states: Communicating and absorbing states; Recurrence and transience;
  • Hitting probabilities and mean hitting times; Birth and death chains, in particular M/M/1 model;
  • 1/2/3-dimensional random walks;
  • Stationary distribution; Convergence to the equilibrium; Ergodic theorem;
  • Markov Chain Monte Carlo methods; Metropolis-Hasting algorithm; Gibbs sampler.



All further information and course material can be found on the learning management system Moodle. If you are interested to attend the course, please ask for the enrolment key at your earliest convenience via email to:

 nguyen [at]

The course will be held in English.

Moodle enrolment



Wednesday 14:15 - 15:45     Starting date: 12 April 2023     Place: Building E2 4, room 1.15, HS IV


Wednesday 9:00 - 10:00 am     Starting date: 19 April 2023     Place: Building E2 5, Seminarraum 1 (U.37).