Statistics 614 – Probability for Statistics

Section 600, Fall 2014, Prof. D. Cline

Welcome!  This page announces and describes my course for the Fall 2014 semester.

This is a course in probability at the measure theoretic level, with emphasis both on understanding measure theory and its relationship to probability and on understanding the crucial roles that probability plays for statistics.  Topics include probability measures, Lebesgue-Stieltjes integration, sigma-fields, random variables, expectation, moment inequalities, independence, convergence of random variables and sample moments, characteristic functions, convergence of distributions, the central limit theorem and the delta method, and conditional expectation.  The intention is to lay a foundation of theory that will enhance the student's ability to read advanced probability and statistics literature and to write a dissertation.

For comments or questions, e-mail me ( or contact the TAMU Statistics Department.

PDF file of the Syllabus (updated 08/13/14)

Course Information

Time and Place: TBA
Instructor: Daren Cline, Blocker 459D, 845-1443.
Office Hours: TBA (My Schedule)
Course Web Page: (this page).  Lecture notes and homework assignments are available at this site.  Access to them will require a password that I will provide to you.
Text: (Required.)
  • A Probability Path, S.I. Resnick (Birkhäuser).
References: (On reserve in Evans Library)
  • Probability and Measure Theory, R.B. Ash (Academic Press).
  • Measure Theory and Probability Theory, K.B. Athreya and S.N. Lahiri (Springer).
  • Probability and Measure, 3rd ed., P. Billingsley (Wiley).
  • Probability: Theory and Examples, 4th ed., R. Durrett (Cambridge Univ. Press).
  • Characteristic Functions, E. Lukacs (Griffin).
  • Probability, A.N. Shiryayev (Springer).
Prerequisite: Statistics 610 and Mathematics 409 (or 615) or their equivalent.
The statistics requirement includes
  • theory of probability distributions for random variables and random vectors,
  • expectations, moments and variance,
  • conditional distributions and conditional expectations,
  • probability generating functions and moment generating functions,
  • probability and moment inequalities,
  • the law of large numbers and the central limit theorem.
The mathematics requirement is advanced calculus, specifically
  • knowing how to produce careful, rigorous proofs,
  • sequences, limits and series,
  • continuity, differentiability and Taylor's expansion,
  • uniform convergence and uniform continuity,
  • integrals and power series, Fourier and Laplace transforms.
A previous course in measure theory is not required.
  • Homework – 30%.  Please see the homework policy below.
  • Midterm Exam – 30%.  Please see the exam policy below.
  • Final Exam – 40%. 
Disabilities Help: The Americans with Disabilities Act ensures that students with disabilities have reasonable accommodation in their learning environment.  If you have a disability and need help, please contact me and Disability Services in B118 Cain Hall, 845-1637.
Academic Integrity: You are expected to maintain the highest integrity in your work for this class, consistent with the university rules on academic integrity.  This includes not passing off anyone else's work as your own, even with their permission.  Please see the homework and exam policies below for specifics.
Copyright: All the resources I provide for this course are copyrighted and may not be copied or distributed without my express, written permission.

Course Policies

Homework Policy: Your homework solutions must be your own work, not from outside sources, consistent with the university rules on academic integrity.  I expect you to follow this policy scrupulously.  Your performance on the exams is much more likely to be better if you do.  (Also, relying on others' solutions will cause me to think I can ask harder questions on the exams!)
You may use:
  • Your textbook and notes from class.
  • Your notes, homework, etc., from a related class that you took or are taking.
  • References listed on the syllabus.
  • Discussion with the me.
  • Voluntary, mutual and cooperative discussion with other students currently taking the class.  This does not mean copying from each other.
You may not use:
  • Solutions manuals (printed or electronic) other than what is provided with the required text.
  • Solutions from previous classes
  • Solutions, notes, homework, etc., from students who took the class previously.
  • Solutions, notes, homework, etc., from classes taught elsewhere or at another time.
  • Copying from students in this class, including expecting them to reveal their solutions in "discussion".  That is, you may work together as indicated above as long as you prepare your own solutions.
Homework is to be submitted by the end of class on its due date unless I specify otherwise.  Late homework is not acceptable.
Exam Policy: Your exam solutions must be your own work, consistent with the university rules on academic integrity.
Each exam will be comprehensive and cumulative.
  • Please bring your own paper.  I ask that separate problems be on separate sheets. 
  • Bring resources (such as notes) only if I explicitly allow them.
I will not expect you to quote theorems and results explicitly but I do expect you to demonstrate that you can make use of them.  Specifically, you will need to:
  • Show all your work.  This does not necessarily mean showing every individual algebraic or calculus step – but it must be clear what those steps would be.
  • Identify (by number, name or description) any theorems, examples or homework problems you use.
  • Verify conditions and assumptions as needed for those theorems and examples.
  • Clearly identify the solution and/or the end of a proof or derivation.
No exam may taken early or made up, except if you provide a university excused absence with appropriate documentation.
Selected problems from my old exams will be available on the course web page.  However, their content may not exactly match this semester's exams.
Makeup Policy: This is based on university policy.
  • If you must miss an exam due to illness or other university excused absence, notify me or the Statistics Department (before, if feasible, otherwise within two working days after you return).  Contact me as soon as possible to schedule a make-up exam.
  • An Incomplete grade will be given only in the event you have completed most of the course but circumstances beyond your control cause prolonged absence from class and the work cannot be made up.

Course Outline

Topic Textbook Section
1. Events and Classes of Events  
  outcomes, events, review of set theory 1.2
  limits of events 1.3, 1.4
  π-classes, fields and σ-fields 1.5, 1.6
  Borel σ-fields 1.7, 1.8
  Dynkin's π-λ class theorem 2.2
2. Probability Measures and Measures  
  probability measures and general measures 2.1 – 2.3
  properties of measures, probability distributions, uniqueness 2.1
  distribution functions, Lebesgue-Stieltjes measures
  extension and existence theorems 2.4, 2.5
3. Random Variables and Measurability  
  random variables and measurable functions, σ-field generated by a random variable 3.1 – 3.3
  induced measures, distribution of a random variable 3.2
  sufficient conditions for measurability 3.2, 5.1
4. Expectation and Integration  
  definitions, consistency of definitions 5.1, 5.2, 5.4
  properties of expectation and integrals 5.2
  Lebesgue-Stieltjes integration, absolute continuity, Riemann integrals 5.6
  monotone and dominated convergence theorems, extensions 5.2, 5.3
5. Advanced Integration  
  equating differently defined integrals 5.5, 5.6
  moments and inequalities, finiteness of moments 5.2
  product spaces, multiple integration, Tonelli-Fubini theorem 5.7 – 5.9
  characteristic functions, inversion formulas 9.1 – 9.5
6. Independence  
  independence of finitely many events or random variables 4.1, 4.2
  infinite collections of independent events or random variables 4.3, 4.4
  Borel-Cantelli lemmas, tail events, Kolmogorov's 0-1 law 4.5
7. Convergence of Sequences of Random Variables  
  modes of convergence and their relationships 6.1 – 6.3, 6.5
  moment inequalities, Jensen's inequality 6.5
  uniform integrability, convergence in mean 6.5, 6.6
8. The Law of Large Numbers and Convergence of Random Series  
  strong law of large numbers 7.1, 7.4
  Glivenko-Cantelli theorem 7.5
  convergence of series 7.3, 7.6
9. Convergence of Distributions  
  weak convergence, Scheffé's theorem 8.1, 8.2, 8.5
  Slutsky's theorem, delta method, Skorohod's lemma, continuous mapping 8.3, 8.6
  tightness, Prohorov's theorem, the continuity theorem 9.5, 9.6
  portmanteau theorem, multivariate convergence 8.4
10. Weak Convergence for Sums and Maxima  
  central limit theorem, Lindeberg-Feller and Lyapunov theorems 9.7, 9.8
  convergence to types, infinitely divisible and stable distributions 8.7
  extreme value distributions 8.7
11. Absolute Continuity and Conditional Expectation  
  conditional expectation 10.2, 10.3
  regular conditional distributions
  conditional independence and exchangeablilty
  absolute continuity, Radon-Nikodym theorem, Lebesgue decomposition 10.1

Lecture Notes (PDF) (Login required)

Lecture Notes: The updated lecture notes will be posted here, as they are ready.

Homework Assignments (PDF) (Login required)

Assignments: Assignments will be posted regularly here.  Please see the homework policy above.  Partial solutions are meant to provide the main ideas, but may not include all necessary details or full justification. They may suggest, however, shortcuts or notational devices that could improve presentation.
For suggestions about what makes a good presentation, please see these homework pointers.  Also, here is a list of questions useful to keep in mind when developing and critiquing your solutions. Ask them habitually of yourself and you will be more likely to avoid common pitfalls.

Exam Information (Login required)

Exams: Information about the exams will be posted here.  Please see the exam policy above.
  • Midterm Exam: TBA.
  • Final Exam: TBA.

Copyright Information

Each document provided on these web pages is copyrighted by me (Daren B.H. Cline) with all rights reserved, whether or not the document explicitly states so.  These documents may only be used for academic purposes and they may not be reproduced, distributed or sold without my permission.  You may refer to them for other classes or for research, just as you would any book.