This is the webpage for Elementary Stochastic Processes (STAT GU4207 / GR5207)
taught at Columbia during Spring 2025. The course an introduction to theory
and applications of stochastic processes. Topics include Markov chains (in
discrete and continuous time), Poisson processes, Gaussian processes,
and Brownian motion, as well as applications to physics, biology, and finance.
Materials: [syllabus],
[lecture notes]
Assignments: [HW1],
[HW2],
[HW3],
[HW4],
[HW5],
[HW6],
[HW7],
[HW8],
[HW9],
[HW10],
[HW11],
[HW12],
[HW13],
[HW14]
Exams: [midterm 1],
[midterm 2],
[final]
Lectures:
- 22 January, [§ 1.1, § 1.2, § 1.3]
- discussion of syllabus,
course motivation, probability review
- 27 January, [§ 1.3, § 2.1]
- some probability exercises, basic definitions
of Markov chains, simple weather example and several representations,
basic of stochastic matrices
- 29 January, [§ 2.1, § 2.2]
- some examples of Markov chains, introduction
to linear algebra of transition matrices
- 03 February, [§ 2.3]
- definition of limiting distribution, examples
of existence/non-existence of limiting distributions, definition
of stationary distribution, limiting distribution implies stationary
distribution
- 05 February, [§ 2.4]
- definition of regular transition matrix, examples of
regularity/non-regularity, conditions for checking regularity/non-regularity,
fundamental theorem of regular Markov chains, calculation of limiting
distribution for lazy Ehrenfest gas model
- 10 February, [§ 2.5, § 2.6]
- definition of reachability, communication, and
communicating class, definition of irreducibility, definition
of recurrence and transience, alternative characterization of
recurrence and transience, infinite success runs and biased random
walk, fundamental limit theorem of irreducible Markov chains
- 12 February, [§ 2.7, § 2.8, § 2.9]
- definition of period and aperiodicity, simple
examples, irreducible and aperiodic implies regular for finite Markov
chains, definition of ergodic, fundamental limit theorem of ergodic
Markov chains, examples of infinite success runs and jump-and-hold chain,
introduction to first-transition analysis and revisiting HW2 Q4
- 17 February, [§ 2.9, § 2.10]
- further discussion of first transition analysis, simple
symmetric random walk exit time from a strip, pattern appearances in coin
flipping, definition of time reversibility, relationship between reversibility
and stationarity, definition of random walk on graph, example of random knight
walk hitting corner
- 19 February, [§ 2.11]
- definition of branching processes, Wald's identity,
calculation of expected generation size, definition of (sub/super)-critical,
discussion of probability generating functions, theorem on extinction
probabilities
- 24 February
- midterm 1
- 26 February, [§ 3.1, § 3.2, § 3.3]
- motivations for continuous-time Markov chains,
discussion of memorylessness, definition of counting process, definition
of Poisson process, simple calculations, law of rare events
- 03 March, [§ 3.4, § 3.5]
- Markov property of Poisson processes, definition of
Poisson process in terms of arrival/interarrival times, alternative solution to
HW6 Q4, expected return time of continuous-time Markov chain, expectation of
multiplicative-return stock price, conditional distribution of arrival
times given number of arrivals, calculation of expected present value of future income
- 05 March, [§ 3.6, § 3.7]
- distribution of first arrival time of M/G/infinity queue,
thinning and superposition, spatial Poisson point process, conditional distribution
of points in spatial Poisson point process given number of points, inhomogeneous
Poisson process, total number of points in inhomogeneous Poisson process
- 10 March, [§ 3.7, § 3.8]
- definition of Hawkes process, distribution of interarrival times,
branching process structure, probability of finitely many total arrivals,
definition of renewal process, renewal theorem, renewal equation, uniform examples
- 12 March, [§ 4.1, § 4.2]
- definition of continuous-time Markov chain and transition
function, examples of Poisson process and two-state chain, Chapman-Kolmogorov
equation, definition of embedded discrete-time Markov chain, construction of
continuous-time Markov chain from independent exponential random variables,
definition of infinitesimal generator, Kolmogorov forward/backward equations,
definition of matrix exponential
- 17 March
- spring break
- 19 March
- spring break
- 24 March, [§ 4.3]
- definitions of continuous-time properties and comparison
with discrete-time properties (limiting distribution, stationary distribution,
irreducibility, transience, positive-recurrence, null-recurrence ergodicity),
discussion of periodicity, not appearing in continuous-time, fundamental
theorem of ergodic continuous-time Markov chains, Jukes-Canor model, reflecting
biased random walk
- 26 March, [§ 4.3, § 4.4]
- continuous-time Ehrenfest gas model, definition of queueing
models, infinitesimal generator of M/M/k queue, limiting distribution of
M/G/infinity queue
- 31 March
- midterm 2
- 02 April, [§ 4.4, § 5.1]
- transience-recurrence characterization for M/G/1 queue in terms of
embedded branching process, review of univariate Gaussian distribution, definition
of Gaussian random vector, definition of discrete-time Gaussian process, examples of
independent Gaussians, Gaussian copies, Gaussian random walk, particle in viscous
fluid, autoregressive processes
- 07 April, [§ 5.2]
- definition of covariance matrix, mean and covariance matrix
for previous examples, definition of Gaussian process on arbitrary index set,
definition of positive semi-definite kernel
- 09 April, [§ 5.2, § 5.3]
- examples of cosine process, linear Gaussian process,
square-exponential process, Ornstein-Uhlenbeck process, definition of
mean-square continuity, characterization of mean-square continuity, definition
of mean square differentiability, sufficient conditions for mean-square
differentiability for stationary Gaussian processes
- 14 April, [§ 5.4]
- review of multivariate Gaussian conditioning formula, conditional
distribution of future position of Gaussian random walk, optimal execution in
Ornstein-Uhlenbeck bridge, conditional distribution of square-exponential process
and its derivative, process-level conditioning formula,
- 16 April, [§ 5.5, § 5.6]
- representing multivariate Gaussian random vectors in terms of independent
standard Gaussians, Karhunen-Loeve representation theorem, examples of cosine process and
Ornstein-Uhlenbeck process, general remarks about convergence of Gaussian process, convergence
of first-order autoregressive process to Ornstein-Uhlenbeck process, convergence of Gaussian
random walk to Brownian motion
- 21 April, [§ 6.1, § 6.2]
- definition of Brownian motion, example calculations with Brownian
motion, Brownian motion as a Gaussian process, discussion of smoothness properties,
definition of Brownian bridge, Karhunen-Loeve representation for Brownian motion
- 23 April, [§ 6.3, § 6.4]
- definition of random walk, Donsker's theorem, recurrence of Brownian
motion, reflection principle for simple symmetric random walk, reflection principle for
Brownian motion, first passage time of Brownian motion, translation-invariance of Brownian
motion, scale-invariance of Brownian motion
- 28 April, [§ 6.5, § 6.6]
- Markov property of Brownian motion, no limiting distribution
of Brownian motion, first transition analysis for Brownian motion, construction of
Brownian motion.
- 30 April, [§ 6.7]
- multidimensional Brownian motion, representation of multidimensional
Brownian motion in terms of independent univariate Brownian motions, location at first passage
time of two-dimensional Brownian motion, definition of geometric Brownian motion, discussion of
relevant properties in mathematical finance, discussion of stochastic differential equations,
Bessel processes, Ito integral