Description

Foundations of Data Science
Prof. C.E. Tsourakakis Assignment 4

Instructions

• No extension will be provided, unless for serious documented reasons.
• Despite having two weeks for this HW, better start early than late!
• Study the material taught in class, and feel free to do so in small groups, but the solutions should be a product of your own work.
• This is not a multiple choice homework; reasoning, and mathematical proofs are required before giving your final answer.
1 MLE and MoM [30 points]
1. (5pts) Let X1,…,Xn be iid Bernoulli(p) samples. In class we sketched the proof that
n the maximum likelihood estimator of. Write a complete proof.
2. (5pts) Assume you have a prior p that is a beta(α,β) and let . Write down the joint distribution of Y,p.
3. (5pts) Let X1,…,Xn be iid N(µ,σ2) where both µ,σ are unknown. What are the MLEs for µ,σ2?
4. (5pts) Let X1,…,Xn be iid Exponential(λ). Find the method of moments estimator for λ.
5. (5pts+5pts) Let X1,…,Xn be iid β(θ,1). Find (a) the MLE and the (b) MoM estimator for θ.
2 To Handshake or Not? [20 points]
3 Mixture of Gaussians [25 points]
Let X,Y be two independent normal RVs, with means µx = 100,µy = 300 and standard deviations σx = σy = 10. Consider the RV U defined by
.
1 of 2
CS365 Spring ’23
Foundations of Data Science
Prof. C.E. Tsourakakis Assignment 4
Alternatively, consider the RV Z that is generated as follows:
(a) With probability we sample Z from N(µ = 100,σ = 100). (b) With probability we sample Z from N(µ = 300,σ2 = 100).

4 Coding EM for Mixture of Gaussians [25 points]
Check the Jupyter notebook on our Git repo.