A Statistician Looks at Affirmative Action in College Admissions

Professor Norm Matloff
University of California, Davis

Purpose: What I Bring to the Table

Hello! I developed this Web page from two interests of mine, Affirmative Action (AA) and statistics. Given the current controversy about the use of AA in college admissions, I believe that the combination of those two interests enables me to present a different perspective on AA than one normally sees in the heated discussions on the topic.

You can read more of my background in my bio. Briefly, I am a professor of computer science, but am also a statistician. I was formerly a professor of statistics, and I conduct research in statistical methodology. By the way my recent book, Statistical Regression and Classification: from Linear Models to Machine Learning was selected for the Eric Ziegal Award in 2017.

Since the lawsuit against Harvard regarding AA is being brought mainly by Chinese immigrants, it is worth mentioning that I have been active in that community for many years, including active in the defense of the engineer Wen Ho Lee accused of spying for China. I am a speaker of Cantonese and Mandarin.

My Own (Nonstatistical) Views

I support AA

To be clear, I am not neutral on AA myself, as I am a proponent of AA. I have for instance chaired my university's Affirmative Action Committee (which deals with faculty and staff, not student admissions). My motivation for this is a deep concern for the well-being of minorities. On top of that, in the college admissions case, I am also very alarmed about what I consider rampant gaming of the system.

Dinner party/employer analogy

I also believe that admissions policy should be Harvard's call. I make the analogy of choosing guests for a dinner party; you want to invite interesting, stimulating people, not necessarily guests with the highest SAT scores.

MIT, for instance, states on its Web page, that it's admissions process is like

...choosing an 1,100-person team to climb a very interesting and fairly rugged mountain -- together.

So if Harvard wants for instance, to favor applicants who overcame adversity, that's their right. No applicant has a "right" to admission, simply because they disapprove of Harvard's policy. (Some critics of AA disagree, pointing out that Harvard receives federal funds. But the federal government has all kinds of AA programs of its own, not only for minorities and women, but also for veterans.)

Another good analogy is that of employers. It is illegal to discriminate on race/gender, but otherwise the government should not be looking over the employer's shoulder and dictate her hiring policy. And again, if she would rather hire someone who overcame adversity even if another applicant has a degree from a fancy school, it is the employer's right.

A similar point holds in terms of the demand by some Asian activists for "transparency" in Harvard's admissions process, i.e. a demand that Harvard lay out the "secret formula" it uses in admissions. But as both sides agree, the process is largely subjective; there is NO secret formula. And you, as an employer say, would not want to be forced adopt a formula for your hiring methods.

Scope of Affirmative Action

Note carefully that AA in college admissions is aimed in obtaining a diverse student body, not only in terms of race but also with respect to gender. The latter is just as important as the former, yet is seldom mentioned in the debates on AA. Keep both aspects in mind here.

One of the goals of AA is to develop role models, and thus transcends mere socioeconomic class. Former Pres. Obama was and is a great role model, in spite of having attended a wealthy private high school. Obama is a graduate of the Harvard Law School.

Note too that though this site focuses on college admissions, AA is far broader than that, such as Minority Business Contracts and Minority Business Loans. Many of these AA policies include Asians. (I will use the latter term for brevity below, but mean Asian-Americans rather than Asian foreign students.)

What This Site Does and Does Not Do

So, Let's Look at a Few Statistical Issues

Group Differences, Covariates

Both expert witnesses, Card and Arcidiacono, agree that the Asian applicants to Harvard differ from other groups. In particular, the Asians tend to have very solid academic credentials, e.g. grades and SAT scores. Indeed, most Harvard applicants have strong academics, so much so that the Harvard admissions office uses the term Standard Strong. The meaning is, in essence, "Applicant has strong academics like everyone else, but not much special otherwise," so it is a negative.

Arcidiacono shows data showing that a disproportionate share of Asian applicants are rated Standard Strong, which he offers as evidence of discrimination. But what matters is whether two applicants, both rated Standard Strong and with similar other characteristics, but one Asian and the other non-Asian, have the same probability of being admitted to Harvard. An example of such an "other characteristic" would be geographical region, say Wyoming. Among Standard Strong applicants from Wyoming, would Asians and non-Asians have the same chance for admission?

These "other characteristics" are called covariates. Much of the disagreement between the two experts revolves around whether the analysis used the proper covariates; frequently one expert criticizes the other for having omitted some important covariate.

The "Higher Bar" Claim

The critics of AA often cite a study by Princeton's Thomas Espenshade that found that Asian-American applicants needed total SAT scores 140 points higher than whites to be admitted to elite schools. The critics interpret this as a consequence of those schools' race-conscious admissions policies.

However, Espenshade himself has warned that this does not necessarily prove there is discrimination against Asians Moreover, the Asian-American Legal Defense and Educational Fund (AALDEF) points out that similar disparities are seen in universities such as UC Berkeley and UCLA, which are legally barred from using race in admissions.

By the way, it is important to distinguish between statistical significance and practical significance. In large studies such as these, even small, unimportant effects can be statistically significant. That 140 figure Espenshade found was statistically significant but is actually not very large. If it is, say, 70 points for each of the Math and Verbal sections, this is a mild effect when viewed in the context of SAT variation, as seen in the next section.

Accuracy and Relevance of the SAT and Other Standardized Tests

Critics of AA view admissions as some kind of contest, akin to, say, the javelin throw. Whoever throws the javelin the furthest "deserves" admission to Harvard etc. I disagree on philosophical grounds, but statistically even those who support the notion need to be aware of the weaknesses of these tests, and in fact weakness of the numbers.

On the surface, the SAT has the virtue of being an objective measure (in contrast to the Personal Rating). Yet it is quite clear that the SAT playing field is far from level.

In addition, though no one is saying the SAT is worthless, its relevance to the admissions process is not entirely clear. The GRE tests, used like the SAT but for admission to grad school, have been shown to have little or no correlation with career success.

By the way, let's clarify that an SAT score is not a "score" in the sense of track and field. It measures the number of standard deviations from the mean (mean 500, standard deviation 100). So if, say, one applicant has an SAT Math score of 720 and another has score 792, it does not mean that the second got 10% more questions correct than the first. (Nor for that matter does it even make sense to add two SAT scores, Math + Verbal, though it is often done, as seen above.)

The Mysterious Shrinking Correlation

Both Professors Card and Arcidiacono agree that the SAT actually plays a very small role in the admissions decision. Indeed, Arcidiacono found that the correlation (actually, logit coefficient) between being admitted and SAT scores, with the other covariates being fixed, was close to 0 (Document 413, p.19). This may seem quite surprising at first, but statistically it makes sense:

As noted earlier, most applicants to Harvard have high scores. Since differences among high scores are not very important (also noted above), at that level the SAT ceases to be much of a factor in admission. As Card notes, "non-academic factors (taken together) explain more than three times as much of the variation in admissions decisions as the academic rating does. That should not be surprising, since exceptional non-academic qualities are less common in the applicant pool than exceptional academic qualities and are thus more likely to distinguish applicants from one another."

The situation is like that of the NBA. In the general population, there will be a substantial correlation between height and basketball talent. But in the NBA, where everyone is big, height has a much lower correlation with success.

Again, recall the term Standard Strong. Harvard applicants typically have strong academic records, so nonacademic aspects are what count most.

Aggregation Fallacies

One way to view the problem of omitted covariates is that it overly aggregates the data. It's like the old joke that if you put one hand in freezing water and the other in boiling water, then on average you feel fine.

One point that Card makes about some of Arcidiacono's analyses is that they aggregate several years worth of admissions data, quite problematic in that admissions criteria change every year. (My side note: My impression has been that universities do this to try to counter gaming of their system.) Aggregating such data is extremely dangerous. Here the covariate is Year. Arcidiacono actually does use Year in some of his analyses, but this can produce quite misleading results if not done properly.

Another example involves Gender as a covariate. Card finds that actually being Asian is a plus among women applicants. This of course may reflect the fact that AA covers not only race but also gender; more on this below.

The Number of Asian Admissions Seems to Have Hit a Plateau

Analysis by conservative writer Ron Unz compared the time trends of Asian enrollment in selective colleges vs. number of college-age Asians, from 1990 to 2011. He found that Asian enrollment at Caltech had tracked the number of Asians in the U.S., but that the Ivy League college Asian enrollments seem to have hit a plateau.

Unz interpreted that as clear evidence of discrimination against Asian applicants to the Ivies, due to race-conscious policies at those schools. However, Caltech also has an AA policy, so that explanation fails. A much better explanation is as follows.

First, the Asian applicants tend to major in STEM fields. For example, at Stanford, reportedly 46% of Computer Science majors are Asian, even though only 23% of the student body as a whole is Asian. This has major (pun intended) implications for the "plateau" effect described by Unz; the Ivies, as liberal arts schools, simply don't have as many slots open for STEM applicants as does Caltech, a STEM school. Asians, a STEM-heavy group, thus would find more difficulty getting admitted to the Ivies -- with no discrimination involved. Again, covariates matter, in this case Major and School Type.

At the same time, it may explain Card's finding that female Asians fare better than their male counterparts. There has been a nationwide concern that not enough women major in Engineering, especially Computer Science (CS). Female Asian students also tend to be STEM-oriented, so all of this may result in university admissions committees being favorably inclined to admit female applicants for, say, CS, including Asian females.

Sample Size Issues

Most of the data analysis done by both sides for this lawsuit have no problems with sample sizes, which are quite ample for statistically valid conclusions.

However, an interesting sample size issue arises in one aspect. The plaintiffs state that Asian applicants get higher ratings from alumni interviewers than what is seen in the Personal Rating. But an important component of the latter is teacher letters of recommendation, which the alumni interviewers do not see. Statistically speaking, a teacher, who has seen the student for an entire year, can provide more accurate information than can an alumnus/a who chats with the student for 15-30 minutes.

Meaning and Quality of the Data, and the Issue of "Merit"

In any statistical analysis, it is crucial to not only "crunch" the data but also to deeply understand what the data mean, and to gauge the quality of that data.

The Personal Rating

Much has been made by the press, and indeed by Prof. Arcidiacono, of the Personal Rating that is part of the Harvard admissions process. A key question then is, What does this rating actually measure?

Arcidiacono found that the Asian applicants tended to have lower scores in Personal Rating. Unfortunately, this has been misinterpreted as some kind of rating of, say charisma. In fact, it is far broader than a personality rating.

Recall my "dinner party" analogy. Just as a highly successful person who grew up in a small town in Wyoming may make an interesting dinner guest, Harvard values this in its student body as well, and so do the other highly selective colleges. Such a factor then becomes part of an applicant's Personal Rating.

From the Card report:

As noted above, family background provides important context for each applicant's achievements. Exhibit 64 and Exhibit 65 (Appendix C) show that the parents of Asian-American and White applicants tend to have different types of occupations. 33% of fathers and 16% of mothers of Asian-American applicants work in the fields of 'Computer and Mathematical,' 'Life, Physical, Social Science,' or 'Architecture and Engineering,' while only 16% and 5% (respectively) of fathers and mothers of White applicants work in those fields. Such differences can reflect not just differences in a family's economic prosperity but also differences in applicants' life experiences. For example, if the son of a professional writer and the son of a police officer display talent in writing, Harvard might regard the latter's talent as more impressive than the former's. The same might be true of the daughter of professional scientists and the daughter of factory workers, both of whom exhibit talent in a scientific field.

Again, this would show up in the Personal Rating -- without any discrimination involved. (Of course, a skeptic may counter that Harvard intentionally chose such criteria as a method of reducing Asian admissions. This charge is not a statistical issue, thus beyond the scope of my document here.)

Many would object that the Asian applicants should not be penalized for having been raised by well-off parents who can nurture them into academic excellence, especially in STEM. But from Harvard's point of view -- again, the "dinner party" analogy -- it is not a matter of "penalizing" anyone, just as someone you might not choose for your dinner event is not "penalized" and you are not being "unfair" in excluding them. Or if you are an employer, you are not "penalizing" job applicants who due to upbringing lack some quality you desire.

It is important to note that the fact that if Applicant A, say, is described as "courageous" in her teacher letters, it doesn't mean that Applicant B, lacking such comments in her letters, therefore must be "cowardly." If Harvard would like a courageous person at its dinner party, it has now found one in A; there is no negative implication about B, who fortunately has never had a situation in which her courage was needed.

IN OTHER WORDS:

The fact that Asian applicants, on average, have lower scores on the Personal Rating category is largely a reflection of the comfortable middle- or upper-class upbringing many of them have enjoyed, rather than some "defect" in their personality.

Much more importantly, the "penalty" argument ignores the enormous familial advantages that many Asian applicants have, such as: Kumon speed math lessons in elementary school; Math Camp in junior and senior high school; SAT coaching; AI-aided strategy for building a re'sume' for college admission, as with IvyMax above; access to university research mentors and schools that groom students to do well in science research contests (see below); etc.

In terms of the title of this section, the above point is clearly a Data Quality issue. Often the high test scores, awards and so on of such applicants come in part from familial advantage. This greatly undermines the claim that these students should be admitted on the basis of "merit."

To be sure, the Personal Rating does include some charisma-like aspects, and critics of Harvard's admissions process suspect that the stereotype of "the quiet Asian" may be part of reason for the lower average score for Asian applicants. On the one hand, there is no denying that, on average, the Asian students do tend to be quieter, as educational researcher Jianhua Feng has explained. On the other hand, unconscious bias of this sort can never be ruled out; it is certainly possible that some Harvard admissions officers could be biased in rating some Asian applicants as "dull dinner partners." (And for that matter, applicants' teacher recommendation letters may be biased in this respect as well.)

Unfortunately, demonstrating this statistically would be extremely difficult. The mere fact that the Asian applicants to Harvard have a lower Personal Rating tells us little. Was this because rather few of them are from Wyoming or are children of police officers? Or was it because the Asian applicants are less charismatic? Or was it unconscious bias? Maybe a combination of all of the above? There simply isn't data to determine this.

Awards, Extracurricular Activities and So On

Much has been made of the fact that in the national high school science contests run by Intel and Siemens, the semifinalists, finalists and winners have been disproportionately Asian (Chinese and Indian). But the implication that the top entrants (of whatever ethnicity) are the nation's best young science talents is highly misleading.

First, one must note that the top entrants tend to come from just a few schools/school districts, largely on Long Island but also in key spots around the nation, that have special programs to groom their students to do well in the contests from Day One. Note, for instance, that the Half Hollow Hills District on Long Island actually has a position titled Academic Research Director. Among other things, these schools link up their students to university researchers, in whose labs the kids work.

Second, typically the work done by the students does not come from their own ideas; they are simply carrying out experiments designed by their university mentors.

From my Bloomberg op-ed (see above link):

During that work [done under the direction of the university researcher], the student will come up with ideas for refinements, but a focus on "their solutions" is exaggerated. Those "High School Student Finds Cure for Cancer" headlines are seriously misleading. Martin Rocek, a university mentor for one Intel semifinalist, recounted for the New York Times how he interacted with the student. Rocek found a "not exceedingly technical" topic in math, gave the student tutorials and suggested the calculations to be done. Professor Miriam Rafailovich, who runs an organized mentoring program for high school researchers at SUNY Stony Brook, told me in an email interview that the contestants "get massive coaching from the schools"... As [a book] and Rafailovich point out, a big motivation for many contestants is to bolster their admission chances to selective colleges. That is a fine goal, but it also explains why many contestants have immigrant parents - who often have a "Harvard or bust" viewpoint. Those kids are more likely to participate.

(The Siemens contest has been discontinued and the Intel reformulated, likely due to concerns similar to those above.)

Knowing the emphasis colleges place on extracurricul activities, many anxious parents have their children engage in a maximal list of such activities. Again this is a Data Quality issue, and the hapless admissions officers must try to divine which activities show "dinner party" quality and which are simply done to build up a re'sume' for college applications.

Role of the Coaching Services

I write the title of this subsection hesitantly, fearing that some may take it that only Asian applicants and their parents game the system. This is absolutely false; people who exploit the system come in all races and ethnicities.

Having said that, here are some details on how services such as the aforementioned Chinese business IvyMax and ThinkTank work, descrbied in an article by Stephanie Ban, concerning CEO Steven Ma. Ban offers insights of the benefit of such services, the special value placed on them by Chinese parents, and even the statistical Data Quality aspects I've discussed here:

Apart from Ma's prices, his so-called guarantee of admission is falling under criticism from admissions officers and internet moguls alike. Stanford's dean of admissions calls Ma's approach "gaming the system..." Ma acknowledges that his program might be putting too much emphasis on getting into top-tier schools, but claims that the main perpetuator of this emphasis is the parents of the children that he works with. Ma boasts a high success rate to back up his assertions about the efficacy of his tutoring... Ma points out that immigrant parents in affluent areas generally view admission to a top-tier school as the primary indicator of success... it seemed like many of ThinkTank's services, like tutoring and guidance on extracurriculars, are available at much lower prices, but perhaps without the option of guaranteed admission. This raises an interesting question: can the qualities that top-tier colleges look for be quantified so precisely [as to be highly predictable]? I think they can, to a point. It's common knowledge that most Ivy [League] students have test scores and GPA's within a certain range and extracurriculars with a certain degree of involvement, but I also think that algorithms can downplay certain qualities that might offset a shortcoming in any category. Qualities like creativity, commitment, and overcoming adversity are touted as paramount to a stellar application, yet they are not easily measured. In fact, for top-tier schools looking to create a well-rounded and diverse class, these "immeasurable" qualities may even take slight precedence over measurable ones... [Ma] already takes care of the measurable criteria in admissions and gives students more opportunities to showcase the immeasurable qualities. Ultimately, Ma is just acting as a knowledgeable (if overpriced) guidance counselor with a mind for statistics. I think admission to a top-tier school will always be a bit of a gamble, but as Ma shows, it helps to have the cards stacked in your favor.
The Alumni Interviews

As mentioned above, in terms of sample size, the alumni interviews are much less statistically reliable than the teacher recommendations portion of the Person Rating. There is also a problem in that an alumnus/a who was a student at Harvard 25 years ago is judging applicants by the standards of that era, when admissions standards were, though high, not nearly as draconian as the current ones.

Claims of Disparate Treatment

Some critics have contended that Harvard's holistic review process, even if well-intended, has a disparate impact on Asian applicants. The problem with that argument is that one could equally note that Harvard policy, basing part of its admissions review on the SAT, AP tests and so on, has a disparate impact on non-Asians. Statistically, the only relevant analysis is to consider the probability of admission, with all covariates other than race fixed.

What about the non-Chinese Asians?

As noted, the vast majority of Asian activists supporting the Harvard lawsuit are specifically Chinese, even more specifically, recent Chinese immigrants. In fact, research by Profs. Karthick Ramakrishnan and Janelle Wong shows that Chinese support for AA has plummeted in recent years, while support among other Asians has risen slightly.

A question of interest is then, Why are the other Asians not opposing AA? I would particularly point to the South Asian parents, who are similar to the Chinese in many respects -- many are immigrants who first came to the U.S. as foreign graduate students in STEM fields. Both of those groups come from countries in which admission to universities is 100% decided on academic factors, specifically a single entrance exam. Why then are the Indians apparently not up in arms like (some) Chinese are? There must be a covariate on which the two groups differ that results in different attitudes on AA. Perhaps it is because the generally more voluble Indians fare better in applying to the Ivies than the Chinese. (See also this summary of polling data. It shows that, consistently across polls, ethnic Chinese are much more opposed to AA than are Indians, Koreans, Filipinos and Vietnamese.)

Wrapping Up

There are serious problems with the general views of critics of AA. Subtle but vital statistical problems pervade the entire topic; data is of poor quality and of meaning at odds with the popular perception; and there are very serious issues of non-level playing fields, quite counter to calls for admitting students on the basis of "merit." There remain philosophical issues that only you yourself can answer, but statistically speaking, the case against AA is very weak.