computed. This is the residual of his actual score from his predicted score and represents the estimated coaching/self-selection effect for that one student. The average of these residuals within the (s,t) cell, denoted by dst, is an unbiased estimate of the parameter dst, which is defined as the average difference between the value of E(Y│X) under a "correct" model for coached students in the (s, t) cell (which may be a different value for each student in the cell) and the value of E(Y X) under the uncoached model (1). If one assumed a strictly additive coaching/self-selection effect according to the model

Ysti = Bo+=1B;Xstii + dst + esti

(4)

where Bo, B1,...,Bp are the same parameters as in the uncoached model, then it would be better to use, instead of dst, a weighted average which would be minimum-variance unbiased under this model. However, as pointed out in Cochran and Rubin (1973, p. 424), dst is an unbiased estimate of dst even when ẞo,ẞ1,...,ẞp take on different values in the coached model.

Tables 2 and 3 show the sample sizes for the uncoached students and the coached students, respectively. Table 3 also shows the number of students who were removed from the analysis of coached students because of rare or extensive missing data patterns. To have included these data patterns in the analysis would have made the computing task much more onerous and was not deemed to be worthwhile.

Table 4 shows the estimated coaching/self-selection effects dst, together with estimated standard errors. These standard errors are calculated on the assumption that model (3) holds for the coached students.

Smoothed Estimates for the Three Coaching Schools and Predictive Distributions for a Future Year

Bayesian methods are recommended to predict the coaching/selfselection effect in a future year. The strongest advantage of Bayesian prediction is that a predictive distribution for a future value can be obtained, which can be made to incorporate uncertainty due to the effects of sampling in the observed data, uncertainty due to year-to-year fluctuation, and uncertainty due to not knowing certain quantities such as variances between schools and variances between years. An additional advantage is that information from schools with large numbers of students can be

used to improve the estimates for schools with smaller numbers of students. (In problems involving larger numbers of groups, a general improvement can be brought about by smoothing all predictions toward the grand mean.) In this section the Bayesian analysis is incorporated into the model and the smoothed estimates of the Verbal and Math coaching/self-selection effects for each school averaged over years is presented. These smoothed estimates also serve as point predictors for a future year. Standard errors for these predictions are also shown. The validity of these predictions and standard errors depend upon the following assumptions:

1. The observed dst are unbiased estimates of the coaching/selfselection effects dst; the standard errors shown in Table 4 represent the uncertainty of these estimates.

st

2. The quantities & within school s for different t, both past and future, are normally distributed about the expected effect d* for that school, which remains constant within the future range being predicted. The variance of the &,, about 8 is the same for each school s. The value being predicted is ds, where t represents a future year.

In the Bayesian analysis, the prior distribution of the expected effects & is a normal distribution. The standard deviation of this distribution and the standard deviation of the 8, within school have independent prior distributions which are each approximately constant over the range of plausible values. The posterior means and standard deviations of the d are shown in Table 5. The means and standard deviations of the predictive distributions for each st (s = A, B, C; t = a future year) for the four dependent variables (juniors' SAT1-V and SAT1-M, seniors' SAT2-V and SAT2 -M) are shown in Table 6.

Interactions between Coaching/Self-Selection Effects and Background Variables

The question now arises: is the coaching/self-selection effect greater for some subsets of the student population than for others? To explore possible evidence of such interactions, a multiple regression analysis of the Verbal and of the Math individual coaching/self-selection effects was performed on the set of variables which had been used to predict the SAT Scores. This was done for the three largest cells of coached students, namely the

SAT2 seniors of School A and the SAT1 juniors of Schools A and B, all from the 1976 peak month administration. Only students with no missing values on background variables were entered into this exploratory analysis; the sample sizes were 102, 103, and 85, respectively. For the two School A cells, the sums of squares and degrees of freedom for multiple regression and for residual are shown in Table 7; the F-values of 1.106 (Verbal) and 1.119 (Math) for seniors and the F-values of 0.606 (Verbal) and 0.735 (Math) for juniors are not significant.

For the School B juniors, the sums of squares and degrees of freedom for multiple regression and for residual are also shown in Table 7; the F-value of 2.112 (Verbal) is significant at the .05 level, while the F-value of 1.587 (Math) is not. To discover which variables contributed most to the analysis of the Verbal individual coaching/self-selection effects, the squared correlation coefficient (r2) was calculated between this dependent variable and each of the predictor variables; these are shown in Table 8. The most significant predictor variable is RACE2 (= 1 for black, 0 for nonblack) with r2 = .0830. The square of the partial correlation coefficient (rACE2) was calculated between every other predictor variable and the Verbal individual coaching/self-selection effect and is also shown in Table 8. The most important predictor variable after RACE2 is entered is INCOME, with r3RACE 2.0664, or r RACE2 = .258. This differs little from the r2 = .0626 (r = -.250) ignoring RACE2, indicating that the effect of self-reported parental income and the effect of black vs. nonblack are essentially separate from each other. Both these effects are statistically significant at the .05 level.

The remainder of this section is devoted to the study of differences in the coaching/self-selection effect across races or minority groups.

Table 9 shows values of background variables and of the individual coaching/self-selection effects for coached students who identified themselves as blacks who took SAT1 as a junior or who took SAT2 as a senior during the peak administration month (and who were thus included in the analysis of this report). There is a strong tendency for both the Verbal and Math coaching/selfselection effects to be above the average (cf. Table 4). For the Verbal effects the pattern is striking: thirteen out of fifteen effects are above average and the remaining two are only very slightly below average.

A significance test was performed between the blacks and whites from Coaching School B, since 13 out of 15 coached blacks

were coached at that school. The variable studied was the Verbal individual coaching/self-selection effect; the null hypothesis was that the mean for blacks equals the mean for whites. Since the overall Verbal averages for the three School B cells (1976 juniors, 1975 seniors, 1976 seniors) were within a fairly narrow range (4.16 to 11.12, from Table 4), the students from these cells were pooled.

The results of this black-white comparison are shown in Table 10. (The data set used excludes 18 students who identified themselves as belonging to other nonblack minority groups and who are among those listed in Table 12.) Equality of population variances for black and white coaching/self-selection effects was not assumed. The Welch approximate t-test was used (see, e.g., Brownlee, 1965, p. 299). The 2-sided test showed significance at the .001 level, with the average for blacks being 46.7 points above that for whites. The average Math coaching/self-selection effect for blacks also exceeded that for whites, but the difference was not statistically significant.

There were five black juniors taking SAT1 and one black senior taking SAT2 at times other than peak administration months, so their scores have not been included in any analysis described so far. A tabulation of their data appears in Table 11. The Verbal effects are higher than the corresponding cell averages in five cases out of six; the average of these six Verbal effects is 52.6 points above the average of the corresponding cell average values. The Math effects are higher than the cell averages four times out of six, averaging 13.6 points above cell average. This pattern is virtually the same as that observed for the blacks who took the SAT in the peak administration months.

Finally, the data for students who identified themselves as belonging to minority groups other than black are presented in Table 12. Both peak-month and nonpeak-month SAT-takers are included. The individual coaching/self-selection effects bear more resemblance to those of white students than to those of black students.

Conclusions

The year-by-year estimated effects for coaching schools A and B, as given in Table 4, are of a similar size to those presented in the FTC report (1979), in that for School A the coaching/self-selection effects range from 20 to 34 SAT points (from 3 to 6.9 standard

errors) and for School B the effects are nonsignificant except for the seniors' SAT2-M in 1975 (26 points = 2.38 standard errors). For School C the estimates fluctuate markedly from year to year, the strongest effect being the juniors' SAT 1-M in 1975, +2.2 standard errors as compared with -1.5 standard errors in 1976.

The smoothed estimates of the school-by-school effects averaged over years, given by Table 5, are only slightly smaller for School A than the unsmoothed estimates. However, the standard deviations shown in Table 5 are larger than the within-year standard errors of Table 4, reflecting the year-to-year fluctuation exhibited in the data. For a future year one would predict a positive coaching/self-selection effect at all schools (Table 6), but with a standard error greater than the value of the predicted effect except for SAT-V at School A (predicted effect = 1.8 standard errors for juniors, 1.3 for seniors) and SAT-V for juniors at School C (predicted effect = 1.3 standard errors).

It is impossible to judge the extent to which each of these effects is due to the coaching program or rather to personal characteristics underlying self-selection of the coached students. However, in a randomized experiment (Alderman and Powers, 1979; Rubin, 1979) which was designed to be free of self-selection effects, special preparation programs offered in eight high schools had estimated coaching effects ranging from 6.3 to 12.5 SAT points, or 0.9 to 1.9 standard errors. The larger estimates derived from the data analyzed in this report are consistent with the possibility that both the motivation to do well on tests, as exemplified by enrollment in a commercial coaching school, and the coaching program itself contribute to a small gain in SAT scores. There is a clear and strong positive difference in the coaching/ self-selection effect of blacks over whites at School B, as shown by Table 10. Whereas the average effect for the 132 whites analyzed here was virtually nil, the average coaching/self-selection effect for the 13 blacks was 46 points, or 4.47 standard errors (S/√N 10.39). One possible interpretation of this phenomenon is that blacks have an initial disadvantage in test-taking that can be overcome by coaching. However, it is also possible that there is a greater difference in personal factors such as motivation between blacks who enroll in a coaching school and blacks who do not than between whites who enroll and whites who do not. We caution that it would be unwarranted to take the relative unimportance of the coefficients for race in the multiple regressions (Table 1) as evidence against the hypothesis that blacks are disadvantaged in test-taking. The strongest predictor variable in the

=