Nearest neighbour averaging

This is one of the smoothing methods. It uses data in the neighbourhood (\(\mathcal{N}\)) to estimate regression function. Works when the dimentionality of the problem is small, \(p \lteq 4\) and with dense training data (large \(N\)). In higher dimensions the neighbourhing data points will be very far, this phenomenon is called the Curse of Dimentionality.

This technique does not work well at the boundaries (extrapolation).

\begin{equation} \hat{f}(X) = Ave(Y|X \in \mathcal{N}(x)) \end{equation}

• Typicall, it is good practice to estimate value of unknown point using 10% training data, that is to say that we need 10% of points in the neighbourhood.