Weighted arithmetic mean
The weighted arithmetic mean is similar to an ordinary arithmetic mean (the most common type of average), except that instead of each of the data points contributing equally to the final average, some data points contribute more than others. The notion of weighted mean plays a role in descriptive statistics and also occurs in a more general form in several other areas of mathematics.
If all the weights are equal, then the weighted mean is the same as the arithmetic mean. While weighted means generally behave in a similar fashion to arithmetic means, they do have a few counterintuitive properties, as captured for instance in Simpson's paradox.
Given two school classes, one with 20 students, and one with 30 students, the grades in each class on a test were:
The mean for the morning class is 80 and the mean of the afternoon class is 90. The unweighted mean of the two means is 85. However, this does not account for the difference in number of students in each class (20 versus 30); hence the value of 85 does not reflect the average student grade (independent of class). The average student grade can be obtained by averaging all the grades, without regard to classes (add all the grades up and divide by the total number of students):
Or, this can be accomplished by weighting the class means by the number of students in each class. The larger class is given more "weight":
Thus, the weighted mean makes it possible to find the mean average student grade without knowing each student's score. Only the class means and the number of students in each class are needed.
Since only the relative weights are relevant, any weighted mean can be expressed using coefficients that sum to one. Such a linear combination is called a convex combination.
Therefore, data elements with a high weight contribute more to the weighted mean than do elements with a low weight. The weights cannot be negative. Some may be zero, but not all of them (since division by zero is not allowed).
Note that one can always normalize the weights by making the following transformation on the original weights:
Using the normalized weight yields the same results as when using the original weights:
When treating the weights as constants, and having a sample of n observations from uncorrelated random variables, all with the same variance and expectation (as is the case for i.i.d random variables), then the variance of the weighted mean can be estimated as the multiplication of the variance by Kish's design effect (see proof):
However, this estimation is rather limited due to the strong assumption about the y observations. This has led to the development of alternative, more general, estimators.
The above formula was taken from Sarndal et. al. (1992) (also presented in Cochran 1977), but was written differently.:52:307 (11.35) The left side is how the variance was written and the right side is how we've developed the weighted version:
In this case, the variability of the ratio depends on the variability of the random variables both in the numerator and the denominator - as well as their correlation. Since there is no closed analytical form to compute this variance, various methods are used for approximate estimation. Primarily Taylor series first-order linearization, asymptotics, and bootstrap/jackknife.:172 The Taylor linearization method could lead to under-estimation of the variance for small sample sizes in general, but that depends on the complexity of the statistic. For the weighted mean, the approximate variance is supposed to be relatively accurate even for medium sample sizes.:176 For when the sampling has a random sample size (as in Poisson sampling), it is as follows::182
A similar re-creation of the proof (up to some mistakes at the end) was provided by Thomas Lumley in crossvalidated.
We have (at least) two versions of variance for the weighted mean: one with known and one with unknown population size estimation. There is no uniformly better approach, but the literature presents several arguments to prefer using the population estimation version (even when the population size is known).:188 For example: if all y values are constant, the estimator with unknown population size will give the correct result, while the one with known population size will have some variability. Also, when the sample size itself is random (e.g.: in Poisson sampling), the version with unknown population mean is considered more stable. Lastly, if the proportion of sampling is negatively correlated with the values (i.e.: smaller chance to sample an observation that is large), then the un-known population size version slightly compensates for that.
It has been shown, by Gatz et. al. (1995), that in comparison to bootstrapping methods, the following (variance estimation of ratio-mean using Taylor series linearization) is a reasonable estimation for the square of the standard error of the mean (when used in the context of measuring chemical constituents)::1186
Gatz et. al. mention that the above formulation was published by Endlich et. al. (1988) when treating the weighted mean as a combination of a weighted total estimator divided by an estimator of the population size.., based on the formulation published by Cochran (1977), as an approximation to the ratio mean. However, Endlich et. al. didn't seem to publish this derivation in their paper (even though they mention they used it), and Cochran's book includes a slightly different formulation.:155 Still, it's almost identical to the formulations described in previous sections.
and the standard error of the weighted mean (with variance weights) is:
The significance of this choice is that this weighted mean is the maximum likelihood estimator of the mean of the probability distributions under the assumption that they are independent and normally distributed with the same mean.standard error of the weighted mean (variance weights, scale corrected)
For small samples, it is customary to use an unbiased estimator for the population variance. In normal unweighted samples, the N in the denominator (corresponding to the sample size) is changed to N − 1 (see Bessel's correction). In the weighted setting, there are actually two different unbiased estimators, one for the case of frequency weights and another for the case of reliability weights.
If the weights are frequency weights (where a weight equals the number of occurrences), then the unbiased estimator is:
Note that the estimator can be unbiased only if the weights are not standardized nor normalized, these processes changing the data's mean and variance and thus leading to a loss of the base rate (the population count, which is a requirement for Bessel's correction).
The degrees of freedom of the weighted, unbiased sample variance vary accordingly from N − 1 down to 0.
The standard deviation is simply the square root of the variance above.
As a side note, other approaches have been described to compute the weighted sample variance.
Similarly to weighted sample variance, there are two different unbiased estimators depending on the type of the weights.
Note that this estimator can be unbiased only if the weights are not standardized nor normalized, these processes changing the data's mean and variance and thus leading to a loss of the base rate (the population count, which is a requirement for Bessel's correction).
For example, consider the weighted mean of the point [1 0] with high variance in the second component and [0 1] with high variance in the first component. Then
which makes sense: the [1 0] estimate is "compliant" in the second component and the [0 1] estimate is compliant in the first component, so the weighted mean is nearly [1 1].