Lably
Skip to content

Kernel regression

  • 2 min read
  • 6 chapters

Related on Lably

Through Statistics, Random variable and Python (programming language)

LabStatisticsLabStatisticsLabStatisticsLabStatistics

In any nonparametric regression, the conditional expectation of a variable Y relative to a variable X may be written:

E⁡(Y∣X)=m(X)

where m is an unknown function.

Nadaraya–Watson kernel regression

Nadaraya and Watson, both in 1964, proposed to estimate m as a locally weighted average, using a kernel as a weighting function. The Nadaraya–Watson estimator is:

m^h(x)=∑i=1nKh(x−xi)yi∑i=1nKh(x−xi)

where Kh(t)=1hK(th) is a kernel with a bandwidth h such that K(⋅) is of order at least 1, that is ∫−∞∞uK(u)du=0.

Derivation

Starting with the definition of conditional expectation,

E⁡(Y∣X=x)=∫yf(y∣x)dy=∫yf(x,y)f(x)dy

we estimate the joint distributions f(x,y) and f(x) using kernel density estimation with a kernel K:

f^(x,y)=1n∑i=1nKh(x−xi)Kh(y−yi),
f^(x)=1n∑i=1nKh(x−xi),

We get:

E^⁡(Y∣X=x)=∫yf^(x,y)f^(x)dy,=∫y∑i=1nKh(x−xi)Kh(y−yi)∑j=1nKh(x−xj)dy,=∑i=1nKh(x−xi)∫yKh(y−yi)dy∑j=1nKh(x−xj),=∑i=1nKh(x−xi)yi∑j=1nKh(x−xj),

which is the Nadaraya–Watson estimator.

Sign in to keep reading

5 more chapters are free with a Lably account.

More in Statistics

Mortality rateDeaths per 1,000 individuals per yearSystematic reviewComprehensive review of research literature using systematic methodsSensitivity and specificityStatistical measure of a binary classificationLongitudinal studyStudy with repeated observations over timePrevalenceNumber of disease cases in a given population at a specific timeStochasticRandomly determined process