Description

Image Gradient
• Consider the image as a function in terms of , ,
• Then by finite difference, then
+ , , − , ,
= = + 1, − ,

• In the exercise, to make it smoother, take the average over two images and the neighbor grids
Lucas-Kanade with Pyramids
• One key step for the algorithm to work is that we want to use the smaller image in the pyramid to reduce the optical motion ➔ estimate the flow for warped image
• For the smaller image, estimate the optical flow
′, , = ′ + ′, + ′, + 1
⇒ , , ≈ + 2′, + 2′, + 1
+ 2′, + 2′ = (, , )
• Then, for the warped image, the optical small is much smaller than the original flow (small motion again!)
Lucas-Kanade with Pyramids (Cont.)
• But what estimated is the optical flow between the warped image and the original image ➔ residual flow
, , = ( + , + , + 1)
• To restore the original optical flow: add back the estimated flow
= 2′ +
= 2′ +
• In fact, this is only the approximation (bonus question: how to get a more accurate result?)

Q: What is the influence of window size and why?
Q: What do you need to change to make it a frame interpolation algorithm?

Horn-Schunk Algorithm
• The overall estimated optical flow should be smooth

Questions