Description

CSE 6363 – Machine Learning
MLE and MAP
1. In class we covered the derivation of basic learning algorithms to derive a model for a coin flip task. Consider a similar problems where we monitor the occurrence of a network event e in a time interval. Assuming that the event’s occurrence is according to a Poisson distribution, we want to learn the best model for the parameter, λ of the distribution given a data set of observed arrival counts. The Poisson distribution for an arrival count k is given by:

We are assuming here that the different data points are independent, i.e. that the time intervals that were used to determine them were independently distributed over a longer time interval.
a) Derive the performance function and the optimization result for analytic MLE optimization for a model learning algorithm that returns the MLE for the parameter λ of the model given a data set D = {k1,…kn}. Make sure you show your steps.
b) Apply the learning algorithm from a) to the following dataset:
D = {2,5,0,3,1,3}.
c) Derive the optimization for a MAP approach using the conjugate prior, the Gamma distribution. The Gamma distribution is:

Note that α and β are constants and that there still is only one parameter, λ, to be learned. Show your derivation and the result for the data in part b) and values for α and β of 2 and 1, respectively.
K Nearest Neighbor
2. Consider the problem where we want to predict the gender of a person from a set of input parameters, namely height, weight, and age. Assume our training data is given as follows:
D = { ((170,57,32), W), ((190,95,28), M), ((150,45,35), W), ((168,65,29), M), ((175,78,26), M),
((185,90,32), M), ((171,65,28), W), ((155,48,31), W),
((165,60,27), W), ((182,80,30), M), ((175,69,28), W), ((178,80,27), M), ((160,50,31), W),
((170,72,30), M), }
a) Using Cartesian distance as the similarity measurements show the results of the gender prediction for the following data items for values of K of 1, 3, and 5. Include the intermedia steps (i.e. distance calculation, neighbor selection, prediction).
(162,53,28),(168,75,32),(175,70,30),(180,85,29)
b) Implement the KNN algorithm for this problem. Your implementation should work with different training data sets and allow to input a data point for the prediction.
c) Repeat the prediction using KNN when the age data is removed (i.e. when only the first two features in the input data are available). Try to determine (using multiple target values) which data gives you better predictions. Show your intermediate results.