Influential observation

In statistics, an influential observation is an observation for a statistical calculation whose deletion from the dataset would noticeably change the result of the calculation.^[1] In particular, in regression analysis an influential observation is one whose deletion has a large effect on the parameter estimates.^[2]

Various methods have been proposed for measuring influence.^[3][4] Assume an estimated regression ${\displaystyle \mathbf {y} =\mathbf {X} \mathbf {b} +\mathbf {e} }$, where ${\displaystyle \mathbf {y} }$ is an n×1 column vector for the response variable, ${\displaystyle \mathbf {X} }$ is the n×k design matrix of explanatory variables (including a constant), ${\displaystyle \mathbf {e} }$ is the n×1 residual vector, and ${\displaystyle \mathbf {b} }$ is a k×1 vector of estimates of some population parameter ${\displaystyle \mathbf {\beta } \in \mathbb {R} ^{k}}$. Also define ${\displaystyle \mathbf {H} \equiv \mathbf {X} \left(\mathbf {X} ^{\mathsf {T}}\mathbf {X} \right)^{-1}\mathbf {X} ^{\mathsf {T}}}$, the projection matrix of ${\displaystyle \mathbf {X} }$. Then we have the following measures of influence:

1. ${\displaystyle {\text{DFBETA}}_{i}\equiv \mathbf {b} -\mathbf {b} _{(-i)}={\frac {\left(\mathbf {X} ^{\mathsf {T}}\mathbf {X} \right)^{-1}\mathbf {x} _{i}^{\mathsf {T}}e_{i}}{1-h_{ii}}}}$, where ${\displaystyle \mathbf {b} _{(-i)}}$ denotes the coefficients estimated with the i-th row ${\displaystyle \mathbf {x} _{i}}$ of ${\displaystyle \mathbf {X} }$ deleted, ${\displaystyle h_{ii}=\mathbf {x} _{i}\left(\mathbf {X} ^{\mathsf {T}}\mathbf {X} \right)^{-1}\mathbf {x} _{i}^{\mathsf {T}}}$ denotes the i-th value of matrix's ${\displaystyle \mathbf {H} }$ main diagonal. Thus DFBETA measures the difference in each parameter estimate with and without the influential point. There is a DFBETA for each variable and each observation (if there are N observations and k variables there are N·k DFBETAs).^[5] Table shows DFBETAs for the third dataset from Anscombe's quartet (bottom left chart in the figure):

An outlier may be defined as a data point that differs markedly from other observations.^[6][7] A high-leverage point are observations made at extreme values of independent variables.^[8] Both types of atypical observations will force the regression line to be close to the point.^[2] In Anscombe's quartet, the bottom right image has a point with high leverage and the bottom left image has an outlying point.

• Influence function (statistics)
• Outlier
• Leverage
  • Partial leverage
• Regression analysis
• Cook's distance § Detecting highly influential observations
• Anomaly detection

• Dehon, Catherine; Gassner, Marjorie; Verardi, Vincenzo (2009). "Beware of 'Good' Outliers and Overoptimistic Conclusions". Oxford Bulletin of Economics and Statistics. 71 (3): 437–452. doi:10.1111/j.1468-0084.2009.00543.x. S2CID 154376487.
• Kennedy, Peter (2003). "Robust Estimation". A Guide to Econometrics (Fifth ed.). Cambridge: The MIT Press. pp. 372–388. ISBN 0-262-61183-X.