Probability and Stochastic Processes

EL6303, Fall 2013
Prereq: Graduate status and MA 3012. *Online version available. 
Professor Ted Rappaport
Wednesdays - 3:00 PM - 5.30 PM
Room: JAB474  

Office Hours: Wednesdays, 11 am - 12 noon or by appointment, 2 Metrotech Center 9th floor

Required TextProbability, Random Variables and Stochastic Processes by Athanasios Papoulis and S. Unnikrishna Pillai

 

Announcements

Posted on December 12, 2013

Lecture notes from 12/11/13

Pillai Lecture notes - 2

Pillai Lecture notes - 1

Posted on December 12, 2013

Final exam room assignments.

Date: 12/18/2013
Time: 3:00-5:30pm
Room Assignments by Lastname:
A-M: JAB474
N-Wang: RH315
Wu-Zhu: JAB678

 

Posted on December 4, 2013

Useful laplace and fourier table, this will be provided on the final exam, along with inside book covers.

LaplaceAndFourier.pdf (Filesize: 865 kb)

 

Approximately 1/3 of the final exam will cover material after exam 2 and 2/3 of the final exam will cover material up to and including exam 2. Three 8.5 x 11 Crib sheets may be used by students; this exam is closed book and notes. The final exam date is December 18, 2013 more details to come.

Posted on November 10, 2013

Exam 2 to be on November 13, 2013, 3pm. GTA George MacCartney shall make the exam room assignments. You may use 2 double-sided 8.5X11” crib sheet. I shall provide tables from the inside book covers. No notes, no calculators, no cellphones and no sharing items allowed.

HW 8 – everyone receives a 100 for this homework and the solution shall be posted by Nov. 11, 2013, before the exam. This material will be covered on Exam 2.

 

Posted on September 25, 2013

Please Note to the class: HW 4 is due BEFORE 2pm on Monday 9/30, and the material will also be covered as part of Exam 1. Submit to the GTA George MacCartney by 2pm in room 9.009 at the end of the help session or submit by uploading on blackboard a scanned PDF following the homework guidelines.  

Posted on September 20, 2013

The first exam will be held on Oct. 2, 2013 in room RH116, 3:00-5:30pm. The exam is closed book, closed notes,no cellphones, no calculators of any kind. Only one double sided 8.5x11” crib sheet allowed. Please bring nothing more than pencils/pens and your crib sheet.

Welcome from Prof. Rappaport

Welcome to all of the new ECE graduate students at NYU Poly! I am very excited to be teaching EL 6303, "Probability and Stochastic Processes", the most important core course in ECE, and I look forward to having you in class!

This course is the fundamental core course for all degrees in ECE, and you must master this material to succeed in graduate school, in research, and in life. I encourage students to buy the book early, and begin reading the book (4th edition US edition) *before* our first day of class. I will hold weekly help sessions on Monday and Friday afternoons, and we will work lots of problems, I will hold extra office hours, and we will have TAs to help everyone master the material, but you will have to work hard. This is a challenging and rigorous class; it is also required class for your program. Come prepared to work and to learn this wonderful field!

Weekly Help Sessions And Resources

To ensure knowledge and learning, will be held on Friday’s and Monday’s every week. Be sure to attend these help sessions to be sure you are learning the material. You want to come to these help sessions as many problems will be worked for practice and understanding!

Help Sessions : 2 Metrotech Center Classroom 9.009, Mondays 12-2PM and Fridays 12 – 2 PM. (There will be no help session on Friday October 4th, 2013.)

In addition to this website please see blackboard for additional resources.

Link to course blackboard page: https://blackboard.poly.edu/

See below for useful notes from the Fall 2011 El6303 lectures:

View EL6303 Lecture notes

Graduate Teaching Assistant (GTA)

George Maccartney's Email: gmac@nyu.edu. 
Office Hours: (Cube 9.120-I) 9th Floor of 2 MetroTech, 12pm-2pm on Tuesdays and Thursdays.

Course Grader

For all inquires regarding the homework grading, please contact the course GTA, George Maccartney. Your course grader is Yuan Zhang.

Yuan Zhang's Email: yz1484@nyu.edu

Exam Solutions

Exam #1:

Please note that the odd answers include the solutions for solving the exam. 

EL6303 Prof. Rappaport Exam one Answer Keys (Even Seats) 
EL6303 Prof. Rappaport Exam one Solutions (Odd seat numbers) 

Exam #2:

The following documents include the answer key for the Odd and Even exams, the worked out solutions by the professor, as well as extra notes from the help session.

Exam2_Answer_Key.pdf
EL6303_Fall2013_Exam2_Odd_Solution.pdf 
EL6303_Fall2013_Exam2_Even_Solution.pdf 

Final Exam:

EL6303 Final Exam Set 1 Solution Dec 26 5pm.pdf

Homework

Homework submission guidelines.

1. All work on 8.5x11 sheets 
2. Cover Sheet with full name registered with Poly (Last, First) (NO NICK NAMES), followed by Poly ID #. Example:

Homework #1
MacCartney, George
Poly ID: 0500000

3. Staple all sheets together 

For online students in section EL6303.1129 follow these additional guidelines:
4. Upload your homework under the assignments section in blackboard
5. Do not submit photos from a camera or phone, the homework must be scanned in.
6. Only submit one PDF file containing all of your work. 
7. The title of the file should follow this example: LastName_PolyID_EL6303_HWX:

MacCartney_0500000_EL6303_HW2.pdf

8. The deadline for submissions is 3:00 pm every Wednesday. Homework will not be accepted after that.

Assignment 1: Assigned 9/4/13; Due 9/11/13

Students may choose to work either Problem 9 or problem 10, you don’t have to work both of these if you do not wish to.

View HW 1 Due 9/11/13

View HW 1 solutions

Assignment 2: Assigned 9/11/13; Due 9/18/13

View HW 2 Due 9/18/13

View HW 2 solutions

Assignment 3: Assigned 9/18/13; Due 9/25/13

View HW 3 Due 9/25/13

View HW 3 solutions

Assignment 4: Assigned 9/25/13; Due 9/30/13 at 2 P.M.

This material will be covered on the exam, submit to the GTA George MacCartney by 2pm in room 9.009 at the end of the help session or submit by uploading on blackboard a scanned PDF following the homework guidelines.

View HW 4 Due 9/30/13 at 2 P.M.

View HW 4 solutions

Assignment 5: Assigned 10/9/13; Due 10/23/2013 at 3pm

View HW 5 Due 10/23/13 at 3 P.M

View HW 5 solutions

Assignment 6: Assigned 10/23/13; Due 10/30/2013 at 3pm

View HW 6 Due 10/30/13 at 3 P.M

View HW 6 solutions

Assignment 7: Assigned 10/30/13; Due 11/06/2013 at 3pm

View HW 7 Due 11/06/13 at 3 P.M

View HW 7 solutions

Assignment 8: Assigned 11/06/13; Due 11/13/2013 at 3pm

View HW 8 Due 11/13/13 at 3 P.M

View HW 8 solutions

Assignment 9: Assigned 11/13/13; Due 11/27/2013 at 3pm

View HW 9 Due 11/27/13 at 3 P.M

View HW 9 solutions

Assignment 10: Assigned 11/27/13; Due 12/04/2013 at 3pm

View HW 10 Due 12/04/13 at 3 P.M

View HW 10 solutions

View HW10 Extra Credit solution

View HW10 Q2 additional solution

 

Grading

Homework must be turned in personally to Prof. Rappaport at the beginning of class. Everyone must do his or her own work without outside assistance.

Grading for the course will be based on:

Homework: 15%

Exam 1 Oct. 2, 2013: 25%

Exam 2 Nov. 13, 2013: 25%

Final exam: 35%

Click here to view the EL6303 Fall 2013 Calendar

Lecture Schedule

DateTopicReading AssignmentsImportant Events in Class
9/4 Induction, Deduction, Relative Frequency, Sets, Repeated Trials, Bernoulli Trials Pages 1- 70  
9/11 Conditional Probability, Bayes’ Theorem and Independent events; Random Variables; Probability Distribution and density functions; Continuous and discrete random variables. Pages 72 -105 HW 1 is due, assigned 9/4/13.
9/18 Functions of one Random Variable and their distributions; Expected value and Variances of a Random Variable. Pages 106-151 HW 2 is due assigned 9/11/13.
9/25

Asymptotic Approximations, Characteristic Functions, Moment Generating Functions, and Higher Order Moments.

Pages 151-165

HW 3 due assigned9/18/13.

9/30

This material will be covered on the exam, submit to the GTA George MacCartney by 2pm in room 9.009 at the end of the help session or submit by uploading on blackboard a scanned PDF following the homework guidelines.

 

HW 4 due assigned9/25/13.

10/2

In-class exam, one 8.5 X 11" crib sheet allowed

  The exam is closed book, closed notes, no calculators of any kind. Only one double sided 8.5x11” crib sheet allowed.
10/9

Joint distribution and density function of Two Random Variables; Independent Random Variables; Jointly Gaussian Random Variables; One function of Two Random Variables and its distribution; Discrete Random Variables and their Functions.

Pages 169-196

HW 5 assigned and due 10/23/13.

10/23

Probability density functions (pdf) of One and Two Function(s) of Two Random Variables - Sum, Difference, Ratio, Product, Magnitude, Phase, Minimum, Maximum, Min/Max etc. – and their joint density functions; the Jacobean, Jointly  distributed Discrete Random Variables and their Functions. Linear functions of Gaussian Random Variables and their joint density functions. Covariance, Correlation, Orthogonality; Uncorrelated and Independent Random Variables; Joint characteristic function and higher order Moments.

Pages 197-219

HW 5 due.

HW 6 assigned due 10/30/13.

Supplement for students regarding Leibniz Rule / Theorem:

LeibnizRule_1.pdf

LeibnizRule_2.pdf

10/30

Conditional distributions and conditional density functions; Conditional Mean and Variance; Conditional Gaussian Random Variables and their Mean and Variances. Stochastic Processes; Concept of Stationarity; Strict Sense Stationary (SSS) and Wide Sense Stationary (WSS) Stochastic Processes; Auto correlation function and its properties;

Pages 220-235

Pages 373-393

HW 6 due.

HW 7 assigned due 11/06/13.

11/06

Stochastic Processes; Concept of Stationarity; Strict Sense Stationary (SSS) and Wide Sense Stationary (WSS) Stochastic Processes; Auto correlation function and its properties; Examples. Stationary Gaussian Process input to Memory-less systems. Discrete Time Processes.

Pages 373-393

HW 7 due.

HW 8 assigned due 11/12/13.

11/13

In-class exam, two 8.5 X 11" crib sheet allowed

 

HW 8 due.

The exam is closed book, closed notes, no calculators of any kind. Only two 8.5x11” crib sheet allowed.

11/20

Systems with stochastic inputs, memoryless systems, strict sense stationary, square law detection, linear systems, output correlation, Response to white noise, differentiators, power spectral density for WSS, Line spectra, moving average filter, differentiations, Hilbert Transform, Wiener-Khinchin Theorem, Least Mean Square Error, Cross Correlation

Pages 393-430

 

11/27

Linear Systems, Factorization for Partial Fraction Expansion, Discrete Time systems, finite order systems, state variables, Fourier Series, Karhunen-Loeve expansion, Spectral Representation of Random Processes

Pages 499-521

HW 9 due.

HW 10 assigned due 12/04/13

12/4

Mean square estimation, Interpolation, Markov sequences, Wiener Filter, Wiener Hopf equation, Predictors

Pages 580-605 HW 10 due
12/11

Approximately 1/3 of the final exam will cover material after exam 2 and 2/3 of the final exam will cover material up to and including exam 2. Three 8.5 x 11 Crib sheets may be used by students; this exam is closed book and notes. The final exam date is December 18, 2013 more details to come.

 

Course Review

 

Course Outline

Continuous and discrete random variables and their joint Probability Distribution and density functions; Functions of one Random Variable and their distributions Independent Random Variables and conditional distributions; One Function of One and Two Random Variables and Two Functions of Two Random variables and their joint density functions; Jointly distributed Discrete Random Variables and their Functions; Characteristic Functions and Higher Order Moments; Covariance, Correlation, Orthogonality; Jointly Gaussian Random Variables; Linear functions of Gaussian Random Variables and their joint density functions. Stochastic Processes and the concept of Stationarity; Strict Sense Stationary (SSS) and Wide Sense Stationary (WSS) Processes; Auto correlation function and its properties; Poisson Processes and Wiener processes; Stochastic Inputs to Linear Time-Invariant (LTI) Systems and their input-output Autocorrelations; Input-Output Power Spectrum for Linear Systems with Stochastic Inputs; Minimum Mean Square Error Estimation(MMSE) and Orthogonality Principle; Auto Regressive Moving Average (ARMA) Processes and their power spectra.

1. Conditional Probability, Bayes’ Theorem and Independent events; Random
variables; Probability Distribution and density functions; Continuous and
discrete random variables.

2. Functions of one Random Variable and their distributions; Expected value and
Variances of a Random Variable; Characteristic Functions, Moment Generating
Functions, and Higher Order Moments.

3. Joint distribution and density function of Two Random Variables; Independent
Random Variables; One function of Two Random Variables and its distribution;
Discrete Random Variables and their Functions.

4. Probability density functions (pdf) of One and Two Function(s) of Two Random
Variables - Sum, Difference, Ratio, Product, Magnitude, Phase, Minimum,
Maximum, Min/Max etc. – and their joint density functions; Jointly distributed
Discrete Random Variables and their Functions

5. Covariance, Correlation, Orthogonality; Uncorrelated and Independent Random
Variables; Joint characteristic function and higher order Moments.

6. Jointly Gaussian Random Variables; Linear functions of Gaussian Random
Variables and their joint density functions

7. Conditional distributions and conditional density functions; Conditional Mean
and Variance; Conditional Gaussian Random Variables and their Mean and
Variances

8. Midterm Exam

9. Stochastic Processes; Concept of Stationarity; Strict Sense Stationary (SSS) and
Wide Sense Stationary (WSS) Stochastic Processes; Auto correlation function and
its properties; Examples.

10. Poisson Processes: Axiomatic development; Probability Distributions of First and
Second arrivals; Inter-Arrival distributions; Distributions of Poisson Arrivals
within the Inter-Arrival/Departure Interval of another Independent Poisson
Process(Geometric Distribution); Wiener Processes; Compound Poisson
Processes.

11. Wide Sense Stationary Processes and their Autocorrelation Functions; Stationary
Gaussian Process input to Memory-less systems. Discrete Time Processes.

12. Stochastic Inputs to Linear Time-Invariant (LTI) Systems; Input-Output
Autocorrelations; Differentiators; Probability of Zero-Crossing for Stationary
Gaussian Processes.

13. Stationary Processes and Power Spectrum; Input-Output Power Spectrum for
Linear Systems with Stochastic Inputs; Quadrature Filtering (Hilbert Transform)
and Single Sideband Modulation.

14. Linear Estimation; Minimum Mean Square Error Estimation(MMSE) and
Orthogonality Principle ; Equivalence of Best Estimator and best Linear
Estimator for Discrete Gaussian Processes; Auto Regressive (AR), Moving
Average (MA), and Auto Regressive Moving Average (ARMA) Processes and their
power spectra.

15.Final Exam

Quick Links

Contact

    NYU WIRELESS
    NYU-Poly Brooklyn Campus 2 MetroTech Center, 9th Fl, Brooklyn, NY 11201

    Leslie Cerve, Administrative Assistant
    Emial: cerve@cs.nyu.edu