Date: Mon, 17 May 2010 00:56:28 -0700 From: Norm Matloff To: Norm Matloff Subject: Mithas/Lucas paper now published To: H-1B/L-1/offshoring e-newsletter (See update at http://heather.cs.ucdavis.edu/Archive/MithasLucas3.txt.--NM) Back in May 2009 I reported on a working paper by Professors Sunil Mithas and Henry Lucas of the Smith School of Business, University of Maryland. You can read my posting from that time at http://heather.cs.ucdavis.edu/Archive/MithasLucas.txt but I will summarize its points, in the process of reviewing the final published paper here, which just came out last week. (Are Foreign IT Workers Cheaper? U.S. Visa Policies and Compensation of Information Technology Professionals, Management Science, Sunil Mithas and Henry C. Lucas Jr., march 2010.) The main finding of the paper is that rather than foreign IT workers (whom I'll refer to here as H-1Bs except when explicitly stating otherwise) being used as cheap labor, they actually make more than Americans. The authors claim that part of this salary premium come from having more years of education and being better skilled, but also that the premium is higher than just education and skills alone would predict. There are severe, fundamental problems with this study: * The data source is invalid, not representative of H-1Bs in the computer field. * There is a lack of adherance to academic standards of even-handedness. I will have four major headings in my posting below: THE ML DATA SOURCE LACK OF EVEN-HANDEDNESS IN THE ANALYSIS THE SKILLS ISSUE THE AUTHORS THEMSELVES ADMIT THAT H-1BS ARE UNDERPAID THE ML DATA SOURCE: This is, in part, what I said in my posting last year: ...the major culprit seems to be Mithas and Lucas' data source. As noted in the article, Mithas and Lucas' analyzed survey data on InformationWeek's readers, who are not representative. The magazine is mainly aimed at managers and nontechnical people affiliated with the IT industry. Its self-description states, "InformationWeek: InformationWeek.com is the industry-leading source of news, analysis, and perspective, serving business technology executives at a cross section of companies." Its job postings section currently lists the following positions: % D. E. Shaw Research seeking Chief of Staff in New York, NY. % Switch and Data seeking Sr Product Marketing Manager in Tampa, FL. % Kadrmas, Lee and Jackson seeking Network Architect in Bismarck, ND. % Osram Sylvania seeking Benefits Specialist in Danvers, MA. % AccuWeather seeking Business Development Manager in Atlanta, GA. Only one of these, the network architect position, is a technical job. ... The basic problem is that InformationWeek is not something that the typical software engineer at, say, Apple or Google would read, much less respond to its survey. And of course this is an absolutely crucial point. Many of you have probably heard of the famous Literary Digest election poll fiasco of 1936, in which that magazine's poll "showed" that Republican presidental candidate Alf Landon would overwhelmingly defeat Democrat Franklin Roosevelt. In the end, Roosevelt was the victor, with 62% of the vote. Why was the Literary Digest poll so far off? The reason was that the population they polled--their own readers, automobile owners etc.--tended to be richer than average, and thus more Republican than average, skewing the results beyond recognition. So, the nature of the population sampled is of the utmost importance. (See my original posting for further problems with the data source.) In a footnote, the authors state the job titles in their sample, some of which are technical, but do not state the percentages for each title. To their credit, Mithas and Lucas did try to address this question of representativeness of the data in their final published paper. They compared various characteristics of people in the survey to the IT population at large, and find similarity. BUT AS YOU CAN SEE, THEIR DEFINTION OF "IT" IS FAR TOO BROAD. One striking illustration of the non-representativeness of the data used by Mithas and Lucas is in their TotalExperience variable. In their data set, the average H-1B worker has 11.23 years of experience! This of course is totally at odds with the USCIS and DOL data. Computer-related H-1Bs have a median age of 27.4; 52% have less than 2 years of experience, and another 41% have 2-5 years. Another illustration: The H-1Bs in their sample had mean wages of $79,000. But the median for H-1Bs in computer-related occupations is about $60,000 (in 2003, the middle of ML's data period), according to the U.S. Citizen and Immigration Services (the former INS), far short of ML's $79,087. Even the 75th percentile in USCIS was only $72,815. The data do vary from year to year, but all of the years of USCIS show that ML's claimed wage figures simply do not jibe with those of the USCIS. Clearly, the H-1BS in the Mithas/Lucas sample were not even close to representative. LACK OF EVEN-HANDEDNESS IN THE ANALYSIS: Incorrect reporting of their results: Mithas and Lucas correctly stress the point that statistical analyses must correct for covariates in order to make two groups comparable. As mentioned above, for instance, they state that in comparing H-1Bs and Americans, one should control for education. Accordingly, in one small section of the published paper, Mithas and Lucas correct for job titles and location (to allow for variations in cost of living). With these corrections, the salary premiums shrink a lot. For instance, where they originally found that H-1Bs get 6.6% (quoted as 6.8% in their remarks to the press) higher salary than U.S. citizens after education and experience are factored in, that premium shrinks down by a factor of almost 3, to 2.6%, once job title and location are controlled for. In other words, their real finding is 2.6%, not 6.6%. Yet in magazine interviews, both last year and now (e.g. InformationWeek and CIO Magazine), the authors cite the 6.6% figure. And when their paper was just a manuscript last year, I asked them permission to discuss this point in my e-newsletter review at the time. They denied my request. This is not the scholarly way to do things. If one's research finds a figures of of 6.6% and 2.6%, with the latter being the result of a more thorough analysis, one should not be highlighting the 6.6% figure to the press. This is a very big point. Given that * linear regression models are only approximations, * less than 2% of ML's data set were H-1Bs, and * most of the job titles were not for technical computer positions that 2.6% figure could easily be negative. Lack of impartial review of previous academic studies: In an academic study, proper use of previous literature is vital. Though the Mithas/Lucas paper does have a very extensive bibliography, its characterization of previous work is very misleading. During that May 2009 period, at Professor Mithas' request, I gave him feedback on a number of aspects of his manuscript, both in the form of my e-newsletter posting and in much more detail in private e-mail. While I must again emphasize that reasonable people can disagree and thus I do not expect him to follow my interpretation of suitability of his data source, he should have corrected the misleading statements that I pointed to in his manuscript. Sadly, he did not do so. For example, in the final published paper, the authors say: Matloff (2003) reviews some studies (mostly conducted by think tanks and research organizations) that do grapple with the impact of H-1B and related visa policies with reference to knowledge workers and IT pro- fessionals, and concludes that firms pay H-1Bs, on average, 15%­33% less than "comparable" U.S. IT professionals (Matloff 2004). However, such a conclusion needs to be viewed with caution because many of these studies, with one or two exceptions, use data prior to 2000 and do not show the "comparability" of H-1B and U.S. professionals on attributes such as education, IT experience, and firm size. Furthermore, they also do not report statistical significance and do not use the types of econometric models with appropriate functional form or control variables that are common in compensation studies (see Ang et al. 2002; Mithas and Krishnan 2008, 2009; Orrenius and Zavodny 2007). This paragraph is highly misleading. It does not mention that in addition to reviewing studies conducted by others, I have done my own studies, which are presented in that 2003 article. Furthermore, my studies do indeed involve data after 2000, and do indeed account for education and experience (the latter through age, a reasonable proxy, one which ML use partly as well). And though my published work does not show the statistical significance numbers, I told Prof. Mithas that I had computed such numbers, and offered to provide them. (More on this point below.) As to "appropriate functional form," I used statistical regression analysis, just like Mithas and Lucas do in their paper. The authors' ignoring of my own studies is no minor point. On the contrary, it is central, as Mithas and Lucas claim that the H-1Bs get higher salaries largely because of their higher education levels and more extensive experience. Since my analysis accounts for these very factors, it becomes key, and the authors should not dismiss it with the phrase "with one or two exceptions." At the very least, they should have told the reader that I did do my own studies and that I did account for variables they consider important, and they should have presented my results--regression coefficients, variables used, and so on. The authors do not even cite the UCLA study, but it is apparently one of the ones they dismiss as being pre-2000. Fine, but if H-1B workers were being exploited in the past, that certainly would suggest at least the possibility that the situation holds today as well, and thus the UCLA paper is very relevant. Moreover, the UCLA study controlled for a ton of variables, far more than either Mithas/Lucas or I did, so given ML's emphasis on controlling for covariates, they should have presented the results of the UCLA study. The authors then say: Some other studies use Labor Condition Applications (LCA) data on "wage rate" and "prevailing wages" from the Department of Labor to claim that average H-1B salaries are lower than the mean annual salaries for these jobs as determined by the U.S. Bureau of Labor Statistics' Occupational Employment Statistics survey of employers (Miano 2005). These LCA-based studies also do not control for education and experience of IT professionals, and many consider them unreliable because they do not use actual salary data, and many approved LCAs may not result in actual H-1B visas (Roman 2006). Actually, I've found, by doing statistical tracking of John Miano's LCA analyses, that LCA data actually does fairly well. If you look for instance at the 75th percentile of LCA salaries, that figure is not far off from the 75th percentile in actual H-1B salaries from other sources, notably the USCIS. I did not mention this to Prof. Mithas, though, so I'm not complaining that he did not include it. But again, I'm more concerned here about responsible, scientific reporting in a journal article, and this one falls very short in the above quoted paragraph. First, I told Prof. Mithas that the only LCA-based studies that have been done are those by John Miano. Language like "some other studies" sounds like the authors are dismissing a broad range of researchers, and is thus misleading. Second, where did that phrasing, "many consider them unreliable" come from? The actual EE Times article the authors cite says "The lower LCA-based pay rates, considered unreliable by some, raise questions about the value of H-1B workers--and of U.S. engineers in general--to their employers, and add fuel to the debate that has long swirled around the H-1B program." So the reporter had merely said "by some," with no attributed source (an academic researcher? an industry spokesperson?), and certainly not "many." Finally, and most importantly, while the Miano studies did not control for education, they did control for experience. I'll explain the latter below, but let's first look at education. The fact that education was NOT controlled is irrelevant to the significance of Miano's findings. After all, Mithas and Lucas are claiming that the H-1Bs are paid more in general, because many of them have more education. If that's true, it should have shown up in Miano's data, which it didn't. In terms of experience, Miano in fact DID account for this, using the DOL levels. Mithas and Lucas seem to have missed Miano's point, which was that the vast majority of H-1Bs are hired at the journeyman level. Indeed, in most of my writings, this has been my main point, that the H-1B program is serving as cheap labor not just because H-1Bs are cheaper than comparable Americans, but also because it gives employers a large pool of younger (thus cheaper) workers to hire in lieu of older (thus more expensive) Americans. Meanwhile, of course, the authors have no problems citing without any criticism whatsoever various pro-industry studies, even for those studies that were never subject to any academic review at all, and even though these studies lack the same things that Mithas and Lucas decry in the studies that disagree with them. For example, take the issue of statistical significance. I'll note in passing that this is a widely misunderstood term; the word "significance" here does NOT mean importance. With a large sample, almost everything is statistically significant, even if there is no practical importance at all. In the large samples of all of these studies, statistical significance basically says nothing. But much more to the point, lots of studies that Mithas and Lucas cite without criticism also do not report statistical significance numbers, a "sin" that Mithas and Lucas claim for the studies that find that H-1Bs are underpaid. To say that Mithas and Lucas are employing a double standard would be a huge understatement. In addition to these (and other) errors of comission concerning the existing literature, the authors make many errors of omission in that regard. In particular, they do not cite the two government-commissioned employer surveys, the 2001 one by the NRC and the one 2003 study by the GAO (both of which I cited in my comments to Prof. Mithas). These surveys actually ASKED employers, "Do you pay H-1Bs less than comparable Americans?", and found that often the employers answered Yes. This is of the utmost importance. Regression analyses, whether done by Mithas and Lucas, Paul Ong (author of the UCLA study), myself or anyone else, are merely approximations, as they come from idealized models that are never exactly correct in the real world. Much more important, one can only INFER causality from regression models; they are only indirect assessments. By contrast, the DIRECT questions asked of employers in the two government surveys ask EXACTLY what we all want to know--are the employers paying less than comparable Americans? (And of course the number of Yes answers would be an underestimate, since most employers who are underpaying their H-1Bs wouldn't admit it.) That fact that many of the surveyed employers openly admitted to paying H-1Bs less than comparable Americans cannot be ignored. Yet nothing whatsoever from Mithas and Lucas on it, a glaring and inexcusable omission from their survey of previous studies. Similarly, there is no mention of the findings of the Tambe and Hitt paper, though it is cited in the bibliography and Tambe is listed in the acknowledgements section (as am I). Again, I found the Tambe and Hitt paper to be seriously flawed, but Mithas and Hitt should have told readers that Tambe and Hitt found that American workers' salaries are going down because of the H-1B program. THE SKILLS ISSUE: It should be noted that although Mithas and Lucas repeatedly say that their data measures skill sets, it definitely does not. What they are actually referring to is the number of years of IT experience. This is a key point, because real skill sets command much more of a salary premium than the 2 or 3% ML found. Remember, the industry claims that the H-1Bs are hired primarily because they have special "hot" skill sets. (ML say this too.) These skills generally command a salary premium of 15-25% (see data in my University of Michigan Journal of Law Reform article), so even if one were to take Mithas and Lucas' 2-3% as accurate, that would indicate that H-1Bs are subject to net UNDERPAYMENT relative to Americans. THE AUTHORS THEMSELVES ADMIT THAT H-1BS ARE UNDERPAID: In my original posting, I had made the following point: ...let's turn to an aspect in which the Maryland study CONFIRMS the previous work on underpayment of H-1Bs. Mithas and Lucas found that green card holders get paid more than H-1Bs of the same education and experience levels. This jibes completely with the central point of the H-1B critics, that H-1Bs, as de facto indentured servants, are exploited in terms of salary. Again to their credit, ML elaborated on this in their final published paper (emphasis added): Possession of a green card provides GREATER BARGAINING POWER and job security for an IT professional compared to someone with a[n H-1B] work visa because (1) employers typically hold work visas, which makes it difficult for an IT professional to easily change his or her employer and (2) work visas are of a limited duration... In other words, there is a fundamental economic reason why H-1Bs are in fact paid LESS than comparable Americans. Norm