CHAPTER 1
INTRODUCTION
Estimation is the art of inferring information about some unknown quantity on the basis of available data. Typically an estimator of some sort is used. The estimator is chosen to perform well under the conditions that are assumed to underly the data. Since these conditions are never known exactly, estimators must be chosen which are robust, which perform well under a variety of underlying conditions.
The theory of robust estimation is based on the specified properties of specified estimators under specified conditions. This book is the result of a large study designed to investigate the interaction of these three components over as wide a range as possible. The results are comprehensive, not exhaustive.
The literature on robust estimation is a varied mixture of theoretical and practical proposals. Some comparative studies are available concentrating primarily on asymptotic theory, including the papers of Huber (19 64), Bickel (1965), Gastwirth (1966), Crow and Siddiqui (1967), and Birnbaum and Laska (1967). In addition, various useful small sample studies have been made, based either on Monte Carlo methods or exact moments of order statistics. Among the former we include Leone, Jayachandran, and Eisenstat (1967) and Takeuchi (1969) while the latter include Gastwirth and Cohen (1970) and Filliben (1969).
As do these authors, we address ourselves to the problem of point estimation of location. The significant advantages of this study are size and scope.
The aims of this survey are, by bringing together a wide experience with a variety of concepts, to (i) augment the known properties of robust estimators (ii) develop and study new estimates.
A great variety of some 6 8 estimates were studied, some well known, others developed during this study; some hand-computable, others requiring a great deal of computer time; some suitable for sample size 5, others best suited to sample sizes over 100. In particular, trimmed means, adaptive estimates , Huber estimates , skipped procedures and the Hodges-Lehmann estimate are among the estimates considered. The estimates are outlined in Chapter 2. Computer programs used to implement these procedures are given in Appendix 11.
The diversity of the survey extends to the variety of properties of both large and small samples which were investigated. Variances of the estimates were calculated for samples of 5, 10, 2 0 and 40. Percentage points were calculated. Breakdown bounds were studied. Hampel's (1968) influence, or Tukey's (1970) sensitivity, curves were calculated to study the dependence of estimators on isolated parts of the data. The ease of computation was investigated. A wide variety of sampling situations were considered, some designed deliberately to test weak points of the procedures considered.
Several new asymptotic results were established to evaluate variances and influence curves. While these were not found for all the estimates in this study, a large number are recorded in Chapter 3. Of particular interest are the formulae for skipped procedures and the unpleasant features of the "shorth".
In the course of this study, W. H. Rogers developed Monte Carlo techniques for sampling from a family of distributions proposed by J. W. Tukey (scale mixtures of zero mean Gaussians) which yielded very great accuracy with a moderate number of samples. The theory used in this study is reported in Chapter 4.
In Chapter 5, numerical results are given for a variety of characteristics. In Chapter 6, these results are analyzed in detail, introducing new methods for the summarization of such a large body of data. J. W. Tukey shouldered this responsibility. Brief additional summaries reflecting the varied experience of some of the contributors are included in Chapter 7.
In the appendices, computer programs for the estimates are given and some material related to the study of robust procedures is included. Appendix 11, as mentioned above, contains the estimate programs used in this study. Appendix 12 contains programs and details for the random number generation of the study. New methods for displaying and assessing variances were developed. The general theory and details relevant to particular distributions are given in Appendix 13. Various numerical integration and Monte Carlo sampling schemes were considered for use in this project. Some considerations and resulting developments are included in Appendix 14. Finally in Appendix 15 are outlined some possible extensions for more efficient Monte Carlo methods.
The results of this study are too diverse to summarize at all completely. We can, however, offer the following guidance for a quick exploration, whether or not this is to be followed by careful study: read Chapter 2 as far as you are strongly interested, then do the same for Chapter 3. Read sections 5E and 5F, then sections 6A, 6B, 6C, continuing as far in six as strongly interested. Begin Chapter 7, continuing as long as interested.
CHAPTER 2
ESTIMATES
2A INTRODUCTION AND INDEX OF ESTIMATES
A great variety of some 6 8 estimates have been included in this study. All of these have the property that they estimate the center of a symmetric distribution (assured by constraint (ii) below). Indeed some were designed to optimize some aspect of this estimation procedure. Other estimates were designed under different conditions or with different objectives in mind. The only constraints on the estimates have been:
(i) all estimates must be represented by computable algorithms,
(ii) all estimates must be location and scale invariant in the sense that if each datum is transformed by x -> ax + b then the estimate T = T(x) is similarly transformed by T -> aT + b.
In this chapter we give brief descriptions of the estimates T([??]) in terms of x(1) ≤ x(2) ≤ ... ≤ x(n), the order statistics of a sample [??] = (xl, ..., xn). The estimates are grouped according to type, e.g., trimmed means, M-estimates etc. with cross references being given at the end of each section.
The descriptions given are not to be taken too literally for the more complicated procedures. The reader who desires certainty rather than a very close approximation should refer to the program as given in Appendix 11.
In those cases in which the originator of an estimate is mentioned without a reference it is to be assumed that the procedures were proposed in the course of this study.
To each estimate there corresponds a number and a mnemonic. For example, the mean, estimate 1, is denoted by M, the median, estimate 6, is denoted by 5 0%. The number corresponds roughly to the order of appearance in this chapter and in the tables. The mnemonics will be introduced as we go along. Exhibit 2-1 gives an index of the estimates indicating where descriptions may be found. The index is ordered by the mnemonic of the estimate using an extension in which the alphabet is prefaced by the digits and some other symbols.
Some estimates form families: the "trims" are 5% to 50%, the "hubers" are H07 to H20, the "hampels" are 12A to 2 5A and ADA, the "sitsteps" are A,B,C.
Many common robust estimates are formed by taking linear combinations of order statistics:
T = a1 x(1) + ... + anx(n).
The condition of location invariance constrains the ai to satisfy
Σ ai = 1.
In general these estimates are relatively easy to compute by hand from ordered data. (The properties of such estimates are easily assessed analytically.) The asymptotic theory of these procedures is well known (cf. Chernoff, Gastwirth3 Johns (1967), for example) and some small sample properties such as moments are also fairly readily computable (cf. Sarhan and Greenberg (1962)).
2B1 Trimmed Means, Means, Midmeans and Medians
The arithmetic mean is a simple, well understood estimate of location. However it is highly non-robust being very sensitive to extreme outliers. One simple way to make the arithmetic mean insensitive to extreme points is first to delete or 'trim' a proportion of the data from each end and then to calculate the arithmetic mean of the remaining numbers. These trimmed means form a family indexed by α, the proportion of the sample size removed from each end.
If a is α multiple of 1/n, an integral number of points are deleted from each end and the trimmed mean is a simple average of the remaining points. If α is not a multiple of 1/n, [αn] points are removed at each end, the largest and smallest remaining points are given a weight p = 1 + [αn] - αn. A weighted mean is then taken.
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]
In this study trimming proportions α = 0, (.05), .50 were used although results for all of these are not reported. The 0 trimmed mean is the usual sample mean, the 0.25 trimmed mean is the midmean. The 0.50 trimmed mean is the median. With the exception of the mean these estimates are defined by their trimming proportion as below.
The estimates of this type included in the study are:
1 M Mean
2 5% 5% symmetrically trimmed mean
3 10% 10% symmetrically trimmed mean
4 15% 15% symmetrically trimmed mean
5 25% 25% symmetrically trimmed mean
6 50% 50% median
Related estimates described elsewhere are:
9 JAE Adaptive trimmed mean (Jaeckel)
10 BIC Bickel modified adaptive trimmed mean
11 SJA Symmetrized adaptive trimmed mean
12 JBT Restricted adaptive trimmed mean
13 JLJ Adaptive linear combination of trimmed means
51 JØH Johns/ adaptive estimate
2B2 Linear Combinations of Selected Order Statistics
Mosteller (194-7) introduced a class of estimates which are simple linear combinations of a small number of order statistics. Of this type and in addition to the median mentioned above, we include an estimate proposed by Gastwirth (1966) involving only the median and the upper and lower tertiles defined by,
T([??]) = 0.3x[n/3 +1] + 0.4(50%) + 0.3x(n-[n/3])
and denoted GAS.
The trimean is another estimate of this form defined in terms of the median and the hinges h1 and h2 which are approximately sample quartiles (see 2D1 and Appendix 11). This estimate is defined by
T([??]) = (h1 + 2(50%) + h2)/4
and is denoted by TRI.
The estimates of this form included in the study are:
7 GAS Gastwirth's estimate
8 TRI Trimean
Related estimates discussed in other sections are:
6 50% Median
41 SST Iteratively s-skipped trimean
42 CST Iteratively c-skipped trimean
43 33T Multiple skipped trimean
48 CTS CTS skipped trimean
2B3 Adaptive Trimmed Means
Jaeckel (1971) proposed a trimmed mean when the trimming proportion, α, was chosen to minimize an estimate of the asymptotic variance of the estimate, T. If the underlying distribution F is symmetric, this asymptotic variance may be estimated by
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]
for αn an integer (see Chapter 3).
The estimation procedure involves choosing the trimming proportion in the range 0 ≤ α ≤ .2 5, to minimize this quantity for αn integral. This estimate is denoted by JAE.
A number of modifications of Jaeckel's adaptive trimmed mean are considered.
(a) Bickel modified the original proposal in two ways. The first modification is that the estimate of the variance of the trimmed mean is based on a pseudo sample of size 2n at each stage. This pseudo sample is obtained by forming residuals from the median and augmenting the sample by the twenty negatives of these numbers. The estimated variance is now proportional to the sample Winsorized variance of the pseudo residuals. The range 0 ≤ α ≤ .25 is still considered but now all α such that 2αn is an integer are tested. The second modification is to bias the trimming proportion down when it is suspected that the choice of optimal trimming proportion has been dictated by a wild fluctuation of the Winsorized variance.
If only the first modification is carried out, the resulting estimate is referred to as the symmetrized adaptive trimmed mean and is denoted SJA. When both modifications are carried out, the resulting procedure is identified as BIC.
(b) A modification of JAE was proposed which would restrict to two the trimmed means considered. This estimate is obtained by calculating and [??]([n/12]/n) and [??]([n/4]/n) and then using the trimmed mean having the smaller of these two (estimated) variances.
This estimate is denoted JBT.
(c) Another modification of the adaptive trimmed mean was proposed by Jaeckel in which a linear combination of two trimmed means is chosen. The estimate is given by
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]
with
c([??]) = (V12 - V22)/(V11 - 2V12 + V22)
where
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]
and for
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]
with α1< α2, estimating the optimum choice where optimum is defined in terms of asymptotic variances and covariances.
This estimate is denoted by JLJ. Unfortunately only the version for n = 20 was programmed.
The estimates dealt with in this section are:
9 JAE Adaptive trimmed mean (Jaeckel)
10 BIC Bickel modified adaptive trimmed mean
11 SJA Symmetrized adaptive trimmed mean
12 JBT Restricted adaptive trimmed mean
13 JLJ Adaptive linear combination of trimmed means
Related estimates discussed in other sections are:
51 JØH Johns' adaptive estimate
2C M-ESTIMATES
In 'regular cases', likelihood estimation of the location and scale parameters θ, σ of a sample from a population with known shape leads to equations of the form
[n.summation over (j=1)] [-f'(zj)/f(zj)] = 0,
and
[n.summation over (j=1)] [zjf'(zj)/f(zj) - 1] = 0,
where f is the density function and zj = (x(j) - θ)/σ.
More generally M-estimates of location are solutions, T, of an equation of the form
[n.summation over (j = 1)] ψ(x(j) - T/s) = 0
where ψ is an odd function and s is either estimated independently or simultaneously from an equation of the form
[n.summation over (j = 1)] χ(x(j) - T/s) = 0.
2C1 Huber Proposal 2
Huber ((1964), page 6) proposed a family of estimates characterized by a function ψ of the form
(a) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]
and the function
(b) χ(x) = ψ2 (x;k) - β(k)
where
β(k) = ∫ ψ(x;k)2 Φ(dx)
These equations (a) and (b) were then solved simultaneously for s and T. The equations were solved iteratively starting with the median and the interquartile range/1.3 5 as initial values for T and s. The mnemonic for these procedures is Huv. Hampel proposed using ψ as above but with s estimated by (med|x(i) -50%|)/.6754. The mnemonic used is Auv. The digits following A and H indicate the value of k.
The estimates of this form included in this study are:
14 H20 Huber proposal 2,k = 2.0
15 H17 Huber proposal 2,k = 1.7
16 H15 Huber proposal 2,k = 1.5
17 H12 Huber proposal 2,k = 1.2
18 H10 Huber proposal 2,k = 1.0
19 H07 Huber proposal 2,k = 0.7
25 A20 Huber M-estimate,k = 2.0,robust scale
26 A15 Huber M-estimate,k = 1.5,robust scale
Related estimates are found in sections 2C2 and 2C3.
2C2 One Step Estimates
These are simple procedures proposed in this context by Bickel (1971) but dating back to LeCam (1956), Neyman (1949) and Fisher which are asymptotically equivalent to estimates of the type proposed by Huber. They are in fact first Gauss-Newton approximations to Huber estimates for fixed scale. To specify one of these estimates we need a parameter k, a preliminary estimate [??] (the mean or the median) and a robust scale estimate s (the interquartile range divided by its expected value under normality). The residuals from [??] are calculated. All observations (bad) whose residuals exceed ks in absolute value are replaced by ks times the sign of the residual; all other observations (good) are left alone. The estimate is then the sum of the numbers so obtained divided by the ^total number of good observations. If all observations are had, the median is used.
These estimates are capable of hand calculation from sorted data.
The estimates of this form included in this study are,
20 M15 One-step Huber, k = 1.5, start = mean
21 D20 One-step Huber, k = 2.0, start = median
22 D15 One-step Huber, k = 1.5, start = median
23 D10 One-step Huber, k = 1.0, start = median
24 D07 One-step Huber, k = 0.7, start = median
27 P15 One-step Huber, k = 1.5, start = median
but using a multiple of the median absolute
deviation from the median as the estimate
of scale, s.
Related estimates are found in sections 2C1 and 2C3.
