Statistics 614 – Probability for Statistics

Section 600, Fall 2008, Prof. D. Cline

Welcome!  This page announces and describes my course for the Fall 2008 semester.

This is a course in probability at the measure theoretic level, with emphases both on understanding measure theory and its relationship to probability and on understanding the crucial roles that probability plays for statistics.  Topics include probability measures, Lebesgue-Stieltjes integration, sigma fields, random variables, expectation, moment inequalities, independence, convergence of random variables and sample moments, characteristic functions, convergence of distributions, the central limit theorem and the delta method, and conditional expectation.  The intention is to lay a foundation of theory that will enhance the student's ability to read advanced statistical literature and to write a disseration in statistics. 

For comments or questions, e-mail me (dcline@stat.tamu.edu) or contact the TAMU Statistics Department.

PDF file of this Syllabus

Course Information

Time and Place: MWF 9:10am–10:00am, Blocker 411.
Instructor: Daren Cline, Blocker 459D, 845-1443.
E-mail: dcline@stat.tamu.edu
Office Hours: MW 1:00pm–2:30pm, or by appointment.  (My Schedule)
Course Web Page: http://stat.tamu.edu/~dcline/614.html
Homework assignments and lecture notes will be provided on this web page.
Text: A Probability Path, S.I. Resnick (Birkhäuser).
References: (to be placed on reserve in Evans Library) Probability and Measure Theory, R.B. Ash (Academic Press).
Measure Theory and Probability Theory, K.B. Athreya and S.N. Lahiri (Springer).
Probability and Measure, 3rd ed., P. Billingsley (Wiley).
Probability: Theory and Examples, 2nd ed., R. Durrett (Duxbury).
Characteristic Functions, E. Lukacs (Griffin).
Probability, A.N. Shiryayev (Springer).
Prerequisite: Statistics 610 or equivalent, and advanced calculus.  In particular, students should be familiar with random variables and their distributions at a practical level and with the theory of functions, integration, limits, etc., and they should be prepared to understand and to provide careful, rigorous proofs.  A course in measure theory is not required since the necessary material will be presented in this course.
Grading: Homework – 30%.  Please see the homework policy below.
Midterm Exam – 30%.  Please see the exam policy below.
Final Exam – 40%. 

Course Policies

Homework Policy: Your homework solutions must be your own work, not from outside sources, consistent with the university rules on academic dishonesty.  I expect you to follow this policy scrupulously.  Your performance on the exams is much more likely to be better.
You may use:
  • Your textbook and notes from class.
  • Your notes, homework, etc., from a related class that you took or are taking.
  • References listed on the syllabus.
  • Discussion with me.
  • Voluntary, mutual and cooperative discussion with other students currently taking the class.
You may not use:
  • Solutions manuals (printed or electronic) and copies of pages from solutions manuals.
  • Solutions from previous classes.
  • Solutions, notes, homework, etc., from classes taught elsewhere or at another time.
  • Solutions, notes, homework, etc., from students who took the class previously.
  • Copying from students in this class, including expecting them to reveal their solutions in "discussion".
Exam Policy: Each exam will be comprehensive and cumulative.
  • Please bring your own paper.  I ask that separate problems be on separate sheets. 
  • Bring resources (such as notes) only if I explicitly allow them.
I will not expect you to quote theorems and results explicitly but I do expect you to demonstrate that you can make use of them.  Specifically, you will need to:
  • Show all your work.  This does not necessarily mean showing every individual algebraic or calculus step – but it must be clear what those steps are.
  • Identify (by number, name or description) any theorems, examples or homework problems you use.
  • Clearly identify the solution and/or the end of a proof or derivation.
Missed Work and Incompletes: This is based on university policy.
  • If you must miss an exam due to illness or circumstances beyond your control, notify me or the Statistics Department, in writing or by email (before, if feasible, otherwise within two working days after you return).  See me as soon as possible to schedule a make-up exam.
  • An Incomplete grade will be given only in the event that circumstances beyond your control cause prolonged absence from class and the work cannot be made up.

Course Outline

Topic Textbook Section
1. Events and Classes of Events  
  outcomes, events, review of set theory 1.2
  limits of events 1.3, 1.4
  p-classes, fields and s-fields 1.5, 1.6
  Borel s-fields 1.7, 1.8
  Dynkin's p-l class theorem 2.2
2. Probability Measures and Measures  
  probability measures and general measures 2.1–2.3
  properties of measures, probability distributions, uniqueness 2.1
  distribution functions, Lebesgue-Stieltjes measures
  extension and existence theorems 2.4, 2.5
3. Random Variables and Measurability  
  random variables and measurable functions, s-field generated by a r.v. 3.1–3.3
  induced measures, distribution of a random variable 3.2
  sufficient conditions for measurability 3.2, 5.1
4. Expectation and Integration  
  definitions, consistency of definitions 5.1, 5.2, 5.4
  properties of expectation and integrals 5.2
  Lebesgue-Stieltjes integration, absolute continuity, Riemann integrals 5.6
  monotone and dominated convergence theorems, extensions 5.2, 5.3
5. Advanced Integration  
  equating differently defined integrals 5.5, 5.6
  moments and inequalities, finiteness of moments 5.2
  product spaces, multiple integration, Tonelli-Fubini theorems 5.7–5.9
  characteristic functions, inversion formulas 9.1–9.5
6. Independence  
  independence of finitely many events or random variables 4.1, 4.2
  infinite collections of independent events or random variables 4.3, 4.4
  Borel-Cantelli lemmas, tail events, Kolmogorov's 0-1 law 4.5
7. Convergence of Sequences of Random Variables  
  modes of convergence, relationships 6.1–6.3, 6.5
  moment inequalities, Jensen's inequality 6.5
  uniform integrability, convergence in mean 6.5, 6.6
8. The Law of Large Numbers and Convergence of Random Series  
  strong law of large numbers 7.1, 7.4
  Glivenko-Cantelli theorem 7.5
  convergence of series 7.3, 7.6
9. Convergence of Distributions  
  weak convergence, Scheffé's theorem 8.1, 8.2, 8.5
  Slutsky's theorem, delta method, Skorohod's lemma, continuous mapping 8.3, 8.6
  tightness, Prohorov's theorem and the continuity theorem 9.5, 9.6
  portmanteau theorem, multivariate convergence 8.4
10. Weak Convergence for Sums and Maxima  
  central limit theorem, Lindeberg-Feller and Lyapunov theorems 9.7, 9.8
  convergence to types, infinitely divisible and stable distributions 8.7
  extreme value distributions 8.7
11. Absolute Continuity and Conditional Expectation  
  conditional expectation 10.2, 10.3
  regular conditional distributions
  conditional independence and exchangeablilty
  absolute continuity, Radon-Nikodym theorem, Lebesgue decomposition 10.1

Lecture Notes (PDF) Login required.

Lecture Notes: The updated lecture notes will be posted here, as they are ready, as will be a list of corrections and minor changes.
Corrections and Changes: as of 11/07/08.

Homework Assignments (PDF) Login required.

Assignments: Assignments will be posted regularly here.  Please see the homework policy above.  Partial solutions are meant to provide the main ideas, but may not include all necessary details or full justification. They may suggest, however, shortcuts or notational devices that could improve presentation.
For suggestions about what makes a good presentation, please see these homework pointers.  Also, here is a list of questions useful to keep in mind when developing and critiquing your solutions. Ask them habitually of yourself and you will be more likely to avoid common pitfalls.
 

Exam Information Login required.

Exams: Information about the exams will be posted here.  Please see the exam policy above.
  • Midterm Exam: Wednesday, 15 October 8:00am–10:00am, Blocker 411.  This exam will cover sections 1.1–5.2 in the notes and Assignments 1–6.  You will be allowed only to bring your lecture notes, with annotations from the lectures and corrections.  No other resources will be allowed.  You may, however, quote results (from memory) that were proved in the homework.
    Sample problems are provided here.   Focus on all the material you need to know from the notes and homework, however, as well as on doing these particular problems.  When you do look at them, think about what results are needed to justify their solutions.

    (10/22/08) Here are complete solutions for the exam.

  • Final Exam: Monday, 8 December, 9:00am–12:00pm, Blocker 411.  This exam will be comprehensive but will focus primarily on the material from chapters 5–11.
    You will be allowed only to bring your lecture notes, with annotations from the lectures and corrections.  No other resources will be allowed.  As in the midterm, you may quote results from the homework by memory, if they are helpful.
    Here are some sample problems.   Focus on all the material you need to know from the notes and homework, however, as well as on doing these particular problems.  When you do look at them, think about what results are needed to justify their solutions.

    (12/09/08) Here are complete solutions for the final exam.

Copyright Information

Each document provided on these web pages is copyrighted by me (Daren B.H. Cline) with all rights reserved, whether or not the document explicitly states so.  These documents may only be used for academic purposes and they may not be reproduced or sold without my permission.  That means you may refer to them for other classes or for research, just as you would any book, as long as neither you nor anyone else reproduces them for sale or other distribution.  If you would like to use some of the material for instruction, you need to first get written permission from me (Daren B.H. Cline, TAMU Department of Statistics, College Station TX 77845-3143).