Description

Homework 3: Binary Classification
1. [15 points] Binary Classifiers
(a) In order to use a linear regression model for binary classification, how do we map the regression output w>x to the class labels y ∈ {−1,1}?
Your answer: y(i) = sign(w|x(i))
(b) In logistic regression, the activation function g( is called sigmoid. Then how do we map the sigmoid output g(w>x) to binary class labels y ∈ {−1,1}?

(c) Is it possible to write the derivative of the sigmoid function g w.r.t a, i.e. ∂a∂g, as a simple function of itself g? If so, how?

(d) Assume quadratic loss is used in the logistic regression together with the sigmoid function. Then the program becomes:
2
g(w
where y ∈ {0,1}. To solve it by gradient descent, what would be the w update equation?
Your answer:
∂f(w)
(j) (j) − α
w := w
∂w
:= w(j) − α(g(w|xi) − yi) · (g(w|xi)) · (1 − g(w|xi)) · x(ij)
where w(j) indicates the jth element of w.
(e) Assume y ∈ {−1,1}. Consider the following program for logistic regression:
min. w
The above program for binary classification makes an assumption on the samples/data points. What is the assumption?
2
Your answer:
The assumption is that samples/data points are i.i.d.(independent, identically distributed).
3