Pedigree and Hiring Decisions

I often wonder how big a role pedigree plays in hiring decisions. I honestly don’t know, and I would like to hear what others think. To better understand the question, it might be helpful to consider a hiring procedure that would, I think, totally eliminate pedigree (as well as gender and ethnicity) as a factor in hiring decisions. I’m not advocating this procedure; I’m just employing it as a device for better understanding how pedigree might affect hiring decisions. So, here’s the proposal: Insist that candidates submit dossiers that contains blinded copies of everything, such that the files that are prepared by the staff and then passed on to the hiring committee members are assigned only a number, containing no information as to the candidate’s name, gender, or associated institutions. The writing sample would, then, have to be prepared for blind review. The C.V. could only list degrees but not the institutions from which they were granted. Teaching evaluations would need to be summarized in such a way that they contain no information about the candidate’s institution. And letter writers would be asked to write in such a way that both their identities and the identities of their institutions would be indiscernible. Only after the hiring committee reviews the files and settles on a list (or ranking) of finalists would someone on the hiring committee be allowed to go back and verify the information, verifying, for instance, that the letter writers were indeed people who knew the candidate in the capacity stated in their letters. This procedure would, then, eliminate the possibility that one candidate might do better than another simply because her graduate program is more prestigious or simply because her letter writers are bigger names in the field.

I suspect that if this proposal were implemented, it would make a huge difference in hiring decisions. And this suggests that pedigree plays a prominent role in hiring decisions. Perhaps, this is as it should be. Perhaps, other things being equal, candidates that come out of more prestigious programs are better candidates. I certainly think that candidates that come out of top-notch programs are better trained than those that come out of bottom-ranked programs. And, perhaps, a glowing letter from a big-named philosopher at a top-notch graduate program is more indicative of a candidate’s excellence than a similarly glowing letter from a not-so-well-known philosopher from a bottom ranked or unranked graduate program.

 

But here’s a worry that I have about pedigree playing a significant role in hiring decisions: the fact that candidates coming out of top-notched programs often perform better (at least in terms of research) than candidates coming out of bottom-ranked programs (even where their writing samples, publication records, and letters of recommendation are otherwise equal) can become a self-fulfilling prophecy. Although pedigree does seem to matter less in hiring decisions as time since the completion of one’s Ph.D. increases, pedigree can continue to affect one’s chances for success even many years after the completion of one’s Ph.D. in that one’s chances for success are significantly affected by the kind of job one lands fresh out of graduate school. A candidate who gets a tenure-track job at a research institution with a ranked Ph.D. program and a 2/2 teaching load has a huge advantage over a candidate who lands a temporary job at an institution primarily focused on teaching with a 4/4 teaching load, all of which are “service” courses—that is, courses such as critical thinking. I know this for a fact, as I’ve had had experiences with both sorts of places.

 

At Cal State Northridge (my first tenure-track job), I had a 4/4 teaching load and taught mainly service courses. Indeed, in the five years that I was there, I had the chance to teach only two upper-division courses. Now, I’m at ASU and I have a 2/2 teaching load (each year: 1 “service” course, two upper-division courses, and one graduate seminar). Surprisingly, I spend almost as much time on teaching as I did at Northridge (if supervising theses counts under teaching), but the difference is that at Northridge the courses I taught didn’t help me learn anything relevant to research. At ASU, by contrast, I learned a lot from teaching upper-division courses, and especially from teaching graduate seminars. And I learned a lot from working with graduate students on their theses. And each year, what I learn helps make me a better philosopher and helps me with my research. Each time I’ve taught a graduate seminar, for instance, I’ve had at least two papers spring from what I learned in teaching them. And I learn a lot from interacting with bright majors and graduate students. Also, at ASU, more of my colleagues are interested in research. This leads to more fruitful discussions with colleagues about philosophy. Discussions that help me become a better philosopher and help me with my research. I also spend a lot less time on service activities at ASU than I did a CSUN. And I find that it is easier to network at ASU – you make more contacts and get more opportunities as you tend to be noticed more once you land a job at an institution with a ranked Ph.D. program. There’s more travel money and better colloquia as well. Lastly, being at an institution that I’m finally happy with means that I don’t have to dedicate time to job searches, something that is quite time-consuming. I was on the job market most every year of my first eight years out of graduate school. I imagine that I could have written at least four papers with the time that I spent on those job searches. If I had stayed at CSUN, I’m convinced that my research would have waned rather than waxed as it has at ASU. So it seems to me that as with money so goes philosophy: the rich get richer while the poor get poorer.

 

And this all leads me to believe that if pedigree affects one’s chances of landing a good tenure-track job right out of graduate school, then it has a profound effect on the rest of one’s career. Of course, it’s possible to scratch and crawl your way out of a less than ideal job, but it’s not easy and the deck is stacked against you. So if pedigree affects your chances of landing a good job right out of the gate, then it also affects your long-term chances for success. So I worry that some talented philosophers fail to realize their full potential do to the influence of pedigree in hiring decisions.

 

So do these observations resonate with others? Do others have other similar or conflicting stories to tell? Does pedigree play too much of a role in our discipline and if so what can and should be done about it?

Comments

[John Turri]

Great post. I think you're absolutely right that small advantages up front can be dramatically amplified. (Malcolm Gladwell's _Outliers_ contains lots of interesting case studies.)

Yes, pedigree plays way too big a role in our profession. It obviously should not be that way, so something should be done about it. What can be done about it is a complicated empirical question. There's lots of related discussion on this thread, though.

[Michael Cholbi]

Doug,

I agree with your sentiments entirely (and not just because I'm at a CSU campus!). Doubtless "pedigree" (which I take you to be using as shorthand for where someone went to grad school and who they studied with) is the largest factor in determining where a philosopher ends up working. What's the largest factor that determines where a person goes to grad school? I'm not sure, but I'm guessing that where one studied as an undergrad is right up there. And what's the largest factor in determining where one studies as an undergrad? Again, not sure, but I'm guessing that contingent factors like the quality of one's secondary education and the educational level of one's parents matter a lot. I think there's a serious tendency to underestimate how this kind of cumulative advantage grows over time and it thereby distorts the allocation of talent, labor, and reward over time.

[Josh Glasgow]

Doug, great stuff. (And, to answer your question: count me as someone who has had a similar experience.) I'm curious, though: why not advocate for "blinding" job dossiers? I share your suspicion about your thought experiment, that removing information about pedigree would make a difference in which schools hire which candidates. If we are right to suspect that, and if we think that pedigree bias is a bad thing, why not implement blind applications? Orchestras long ago went to "screened" auditions, where musicians audition from behind a screen, which had the effects of both revealing how much gender bias had existed and moving women musicians into prominent roles. It would seem like blinding applications would be a relatively easy way to accomplish something similar in academia. Even those who deny the existence of pedigree (or gender or race or...) bias could get behind this proposal, since they should see it as a quick and decisive way to prove their point.

Is your reluctance to advocate blinding applications based in the judgment that pedigree does actually indicate a likelihood of better training? I suspect that some people don't think that pedigree bias is a bad thing, in many cases for reasons like that. But I think that even if the better-pedigree-indicates-a-likelihood-of-better-training thesis is true, we should ask ourselves how much stock we want to put in that likelihood when we're hiring. Even if it's true across the board that better pedigreed programs tend to better train their students, we're (obviously) hiring individual candidates, not statistical profiles of PhD programs. So shouldn't we want to see some independent proof that the probably-better-trained candidate in question is actually better than the probably-not-as-awesomely-trained candidate? And if so, why not just make that independent evidence (be it recommendations, writing samples, teaching evaluations, or whatever) the decisive factor?

Appealing to pedigree-independent evidence might fail to be useful if there were many cases where candidates appear to be equally qualified other than pedigree, but how often does that happen? Also, if the better-training thesis is true, and if you're choosing between two candidates who are equally qualified other than coming from a generally-better-training program and a generally-not-as-good-training program, wouldn't it be wise to choose the candidate with worse pedigree? After all, that candidate shows a greater ability to do the same level of work without the help of better training. I haven't really thought all of this through, and I'm very curious to hear what I'm leaving out of the equation, but these points seem to suggest that we should remove pedigree information from applications until we have to choose between all-else-equal candidates, and then at that point choose the one with the less impressive pedigree. (That's surprising, I think.)

[Mark Eli Kalderon]

While this may resonate with some, I have serious reservations about the main drift of the argument here. Two observations.

First, you don't seriously believe that where someone was trained and who their teachers were has *no* impact on their philosophical development. Well then, you are proposing to intentionally dispose of some (admittedly weak and defeasible evidence). In hiring situations, the problem typically faced is lack of evidence. This seems counterproductive.

If you look at junior hires at top departments, they are not uniformly from top departments. This is evidence that pedigree is not playing an undue role in influencing hiring decisions.

If we are to take this at all seriously, we need a lot more evidence than you provide. And thought experiments about the outcomes of blind reviews is not evidence. It is the expression of antecedent sentiment.

[Richard]

"It’s possible to scratch and crawl your way out of a less than ideal job, but it’s not easy and the deck is stacked against you. So if pedigree affects your chances of landing a good job right out of the gate, then it also affects your long-term chances for success. So I worry that some talented philosophers fail to realize their full potential due to the influence of pedigree in hiring decisions."

That's fair worry, but the big question is: would any alternative system do any better? If pedigree was ignored, then some other talented philosophers would fail to realize their full potential, due to the (increased) incompleteness of the evidence on which hiring decisions were made.

[Jamie]

I’m skeptical about blind dossiers.
Assume the applicant is applying from graduate school. She can’t list the seminars she has taken. (I bet you I can pick out a list of NYU seminars from a line-up.) Her referees cannot compare her to other students. When I read the referees’ letters, I will have no idea how reliable the writers are. If a letter says the student is among the best the writer has ever seen, I will have no idea what that means. The writing sample cannot be something published. And if there is even a list of courses she has TAed and taught, that will reveal a lot of information (for instance, a long list indicates the degree is probably from a state university). Now assume the applicant is not coming straight from graduate school. She cannot say what jobs she has held. She cannot say anywhere in the dossier that she held an Arche fellowship, or an ANU fellowship. She cannot list any awards she has won.

When I read this dossier, I think I’m going to have to make my decision based entirely on the writing sample, which has a good chance of not being the candidate’s best work (because she published her best paper). That’s going to make the decision extremely difficult, and extremely time-consuming.

In case this isn’t clear: I am not expressing skepticism about the idea that there is a problem that blind dossiers might help to address. (I don’t really know whether there is; I wouldn’t be surprised if it turned out there is.) I am expressing skepticism about the feasibility of the solution.

[Ben Bradley]

I'm wondering how exactly you guys are thinking that blind recommendation letters are supposed to work. Part of the information you get from a letter comes from who is doing the writing. If I read an anonymous letter, I have no context for claims such as "X is the best student I've had in 5 years" and the like. And I have less reason to trust the letter's content, since there is no incentive for the writer to exercise restraint in praising the candidate.

If you really wanted to get serious about not knowing the pedigrees of job candidates, you'd also have to make sure to avoid attending conferences (or reading conference websites) so that you wouldn't find out that candidate X wrote paper Y and thereby learn pedigree information. I guess you'd also have to avoid reading journals. (Or, you could remove writing samples, diss abstracts and paper titles from the files too. I'm starting to like this- the files would be so small!)

This is on top of the enormous amount of work that would be involved to do all this blinding. So I guess the point is that there's not going to be a good way to eliminate pedigree information from candidate files even if it would result in better decision-making (I don't know whether it would or not). Trying to emulate the practices of orchestras is a good idea in principle but there are way too many differences.

[Ben Bradley]

OK, I typed too slowly and Jamie beat me. Damn.

[meghan]

Somewhat an echo of the previous posts...

Suppose that you only know 2 things about a particular candidate: (1) she has just finished a Ph.D., and (2) Kit Fine has written a letter in which he describes her as an exceptionally deep and original metaphysician. If that candidate is compared with another newly minted Ph.D. who is praised just as highly by a rather middling senior philosopher, shouldn't the former clearly come out ahead?

I just can't see how we can avoid acknowledging that, head to head, assessment letters written by brilliant philosophers should carry (far?) more weight than such letters by lesser philosophers.

[Josh Glasgow]

Jamie and Ben, I think your worries about actually generating useful blind dossiers are generally on target. But I also suspect that pedigree can be made more or less salient, and that we can do some things to make it less salient. (I'm just going to assume here that pedigree bias is bad.) One easy thing to do would be to read a candidate's CV only after her pedigree has been marked out. Then one can absorb all of the candidate's accomplishments without pedigree setting the first impression. Regarding knowing the candidate's pedigree from indirect sources (publications, conferences, currently held jobs or post-docs, etc.), I think there are a couple of things to remember here. First, there are many, many jobs where those in a position to hire either have not kept up on the literature over the years or are ignorant about the AOS for which they are hiring, and who have no idea who's held recent post-docs at ANU. Second, everyone else can surely avoid the temptation to try to find out the pedigree of candidates before forming first impressions. Obviously, there is no way to do that for some candidates (such as ones whose work one already knows well). Even then, though, again it seems that there would be value in letting something besides pedigree set one's very first impression by structuring CVs in a certain way. (How about just making the first page of the CV, with name, address, institution, and pedigree the last page of the CV? Even that little change might make a difference, and it hardly seems to do much harm.)

Mark, is "you" me? Just to be clear, I never said that training has no impact on one's development. (I don't think anyone else did, either. I'd imagine that nearly all of us have been frustrated and amazed by the difference between those who can and those who cannot mentor and teach and model well and by the impact the difference has on students.) The point in disposing of whatever evidence of quality pedigree provides (were it possible to eliminate!) would be to counteract the bias that it creates, on the hypothesis that the bias overpowers useful information that is provided by other components of the dossier and on the assumption that pedigree itself provides comparatively little (if some) useful information. Also, the rest of your comment seemed to suggest that you think there isn't much (or any) pedigree bias; is that right? I'd be curious to hear more about why--while we all know of a couple of exceptions, how exactly do you see the numbers being evidence against the existence of pedigree bias? (And why only consider top departments? I thought the question was about a general hiring bias. By the way, for what it's worth, I know of some schools that are consciously biased against at least some top programs, because they are perceived as not training teachers who are immediately ready to teach at a quality level as high as less-well-pedigreed programs.)

[Clayton Littlejohn]

Doug,

You won't be surprised to know that I agree with much of what you say here. That being said, there seem to be serious practical difficulties in implementing the blind review. One strategy would be to do initial screening on the basis of blinded information and then do the letters next to winnow down the applicants to determine who gets interviews. That might limit the impact of pedigree. It's not ideal, but it might be an improvement.

As for the letters themselves, I'm worried about Meghan's remarks. I sort of agree that if you had two candidates that had _nothing_ on a CV pedigree and letters could matter. That being said, I'd want to know why there was nothing on their CV. And, once there were publications and conference presentations on their CV, I'd think that the letters would carry less and less weight. I don't imagine that we're in a market where there was a paucity of candidates with blind-refereed publications looking for TT jobs. Anyway, three worries about the letters. First, it's difficult to put letter writers on a scale. Second, even if we overcome that difficulty, I suspect that many letters ought to be taken with a grain of salt. (Are letter writers really assumed to be objective?) Third, suppose we think there's a correlation between being a brilliant philosopher and being a philosopher employed by a fancy program. There's a good chance that a philosopher at a lesser program writing a letter has had experience with students from the top end of the Leiter scale (i.e., their fellow graduate students) and those at the lower end of the Leiter scale (i.e., the students that they supervise). I think they are in a better position to assess the relative merits of the candidates on the market than someone who came from a fancy program and has always worked in such a program.

[Full disclosure, I'm biased. I came from a program that wasn't ranked and teach at least 4 courses per semester.]

[David Sobel]

My first reaction was much like Jamie’s. Making the letters anonymous would lose a lot of valuable information. I agree that people can give too much weight to such things but then they can give too much weight to other things too. Do people give too much weight to pedigree more than they give too much weight to other things? I’ll guess yes until the APA stage when I bet quickness, nerves, and interpersonal skills start getting lots of weight. One issue here is that letter writers, in my experience, see themselves as advocates for the candidate and say the nicest things they can without feeling like they are lying (rather than seeing themselves as an advocate for the profession and seeking to make sure the best people get hired). Everyone playing by these rules gives an extra advantage to the candidates whose letter writers are most trusted. Should this change? I don’t know. There is a real advantage to the atmosphere created when you think your mentors are rooting you on, at least in such contexts.

[Doug Portmore]

Josh:

I'm curious, though: why not advocate for "blinding" job dossiers?

Because I haven’t thought about it enough and because I have concerns about its feasibility. First, I need to think more about whether pedigree has much of an impact and whether that impact is warranted. I’m not against blinding applications either. I’m undecided.

Also, if the better-training thesis is true, and if you're choosing between two candidates who are equally qualified other than coming from a generally-better-training program and a generally-not-as-good-training program, wouldn't it be wise to choose the candidate with worse pedigree? After all, that candidate shows a greater ability to do the same level of work without the help of better training.

A very interesting point and one that had crossed my mind as well. I think that if one candidate wasn’t as well trained as another, the one with poorer training can fairly easily make up for that poor training by doing what professional philosophers normally do. However, there’s nothing one can do to make up for less talent. So I suspect that what you’re suggesting is right. If two candidates have blinded dossiers of like quality, then the one with worse training must be more talented.

Mark:

you don't seriously believe that where someone was trained and who their teachers were has *no* impact on their philosophical development.

Not only don’t I think that, but I even explicitly said as much in my post. I said: “I certainly think that candidates that come out of top-notch programs are better trained than those that come out of bottom-ranked programs.”

If you look at junior hires at top departments, they are not uniformly from top departments. This is evidence that pedigree is not playing an undue role in influencing hiring decisions.

It’s evidence that pedigree isn’t playing a decisive role in every hiring decision. It’s not evidence that it’s not playing an undue role in influencing some (or even many) hiring decisions. We need to keep in mind that most hiring decisions are not made from top departments just because most hiring departments are not top departments. My concern is that pedigree is having an undue influence in many hiring decisions, not that it is having an undue influence at the top department. It may be that there’s a pedigree bias and that that bias plays a bigger role as the quality of department goes down.

If we are to take this at all seriously, we need a lot more evidence than you provide.

If ‘this’ refers to the thought that pedigree might be playing an undue role in hiring decisions, I agree that we need a lot more evidence that what I’ve provided.

Jamie,

I share many of your worries about the feasibility of blinded dossiers. But, of course, no system of blind review is perfect. For instance, I often know the identities of the authors whose papers I’m asked to review, because I try to know who’s doing what in my areas of research. So blind-review can only be blind to a point. Perhaps, it should be made more blind than it is. I’m not sure that even this is feasible, but I’m not convinced that it couldn’t be more blind than it is (which is not blind at all) and to good effect.

When I read this dossier, I think I’m going to have to make my decision based entirely on the writing sample, which has a good chance of not being the candidate’s best work (because she published her best paper).

Why can’t the candidate turn in a blinded version of his or her best published work? Sure, some philosophers will recognize it from having seen it in a journal but many won't. Again, it is not possible to ensure that blind review is totally blind for everyone and no one is suggesting that we should shoot for what's not possible.

Ben:

I share your worries about blinded letters. But I’m not so sure how reliable evidence from letters are in the first place. I tend to give more weight to a glowing letter from a big-named philosopher than I do to a glowing letter a no-named philosopher, but perhaps the big-named philosophers are no more reliable than the no-named philosopher in writing letters that accurately reflect the candidate’s true potential.

If you really wanted to get serious about not knowing the pedigrees of job candidates, you'd also have to make sure to avoid attending conferences (or reading conference websites) so that you wouldn't find out that candidate X wrote paper Y and thereby learn pedigree information.

Isn’t that true also of the blind review of journal articles. I don’t think anyone would suggest that we get that serious about the blind review of anything. So I think that you’re attacking a straw man there.

Meghan:

I just can't see how we can avoid acknowledging that, head to head, assessment letters written by brilliant philosophers should carry (far?) more weight than such letters by lesser philosophers.

Why is that? Is there good evidence that brilliant philosophers are better at identifying talent than middling philosophers are? I consider myself a middling philosopher, but I think that I can recognize a talented graduate student just as well as the best of them. And is there good evidence that the depictions of talent that brilliant philosophers provide in their letters are more accurate and honest than those of middling philosophers? I’ve found in my experience on search committees that some brilliant philosophers don’t give very accurate depictions of a candidate’s potential. Of course, this is poor evidence. My claim is that we don’t have good enough evidence either way.

[Jamie]

Doug,

First: the reason I think it's important to know the identities of letter-writers isn't that a big-named philosopher is a better judge of talent. It's that I can calibrate the praise, and I know what it means if the writer says "this is one of the five best students I've seen in the past twenty years." There are some very big-named philosophers whose praise of a student I discount by, well, approximately infinity. And I suppose we aren't doubting that being one of the best five students to pass through Pittsburgh's philosophy department in the past twenty years is more impressive (in the normative sense) than being one of the best five to pass through Washington University (trying to pick a program that I like a lot and that I bet provides excellent training but just doesn't have Pitt's track record).

You think maybe letters aren't worth much. Published papers, you suggest, could be fixed up so that the publication information is removed. I wonder what you would suggest the search committee use to make its short list. I imagine you'll say, they should read the writing samples. But I'll let you speak for yourself...

[Doug Portmore]

Jamie:

These are all fair points. I agree that if letters are made blind some useful information will be lost. My concern is over whether non-blinded letters might have, on the whole, more of a bad influence than a good influence in hiring decisions. I imagine that when doing a search for a position at Brown, you get lots of letters that say something really useful, such as "This is best student out of Princeton in the last five years." This is less true when you do a search for a position at State Teaching University. The best students out of Princeton often don't even apply to such positions, which often have a 4/4 teaching loads. Many of the letters that you see on search committees at teaching institutions don't say anything like this is the best student in x years.

Since no one can or should force all departments to request blinded dossiers, we should focus on whether any sorts of departments should do so, and, if so, which sorts of departments should do so.

Published papers, you suggest, could be fixed up so that the publication information is removed. I wonder what you would suggest the search committee use to make its short list. I imagine you'll say, they should read the writing samples.

The search committee could make the decision based on the blinded teaching evaluations, the blinded letters, and the blinded CVs, which could indicate how many years of teaching experience the candidates have as well as at what sorts of institutions (e.g., small liberal arts college, elite ivy league school, large state university), and it could indicate how many articles/books have been published and with what journals and presses.

Again, I share many of your worries about the feasibility of blinded dossiers and am consequently undecided as to whether it's a good idea. And I'm finding that this kind of discussion is useful in seeing the merits of both sides.

[Alex Gregory]

Doug,

This is an interesting discussion. I have just a small question. You wrote:
"The search committee could make the decision based on [...] blinded CVs, which could indicate [...] how many articles/books have been published and with what journals and presses."

Presumably you'd be interested to know which journals and presses their publications were with because different journal and presses have different standards for what they let through. But isn't that exactly the same as with institutions? That is, what's the relevant difference between this kind of pedigree and other kinds?

(I have no view here, I'm just curious!)

[Doug Portmore]

To give people a better idea of what I have in mind, here's a blinded version of my cv: http://www.public.asu.edu/~dportmor/cv blinded.pdf.

[Doug Portmore]

Alex,

I take it that we should assess professional success along three dimensions: research, teaching, and service. To illustrate, I'll focus on research. When it comes to tenure and promotion decisions with regard to research, what's relevant is the quantity, quality, and influence of one's published work. The fact that a candidate wrote a dissertation under big-name philosopher x at Leiterific program y does not in itself demonstrate that this candidate will publish a sufficient number of articles or books that are of sufficient quality and quantity. In fact, it doesn't demonstrate that the candidate will publish at all. The fact that a candidate has published x number of articles and books in y number of years in journals and presses of z influence does demonstrate that the candidate has a certain level of productivity and influence. And the quality of the writing sample and of the journals and presses in which the candidate has published should provide a much more reliable indicator of future research success than pedigree does. The only reason it wouldn't is that, perhaps, some top-notch departments discourage their graduate students from publishing before finishing their Ph.D.s. But this has already started to change and will change much more rapidly if a number of departments start requesting blinded dossiers. The idea is that pedigree isn't relevant to promotion and tenure decisions so we should be cautious of giving it too much influence in hiring decisions.

[Greg]

The authors of many of the above comments worry that even though pedigree may have undue influence on hiring decisions, switching to blinded dossiers would result in the loss of important information (see for instance, the comments of Jamie, Ben Bradley, and Meghan). I'd like to suggest a way to address this worry.

Suppose that instead of having every member of the hiring committee receive blinded dossiers, half of the hiring committee looked at blinded dossiers and the other half looked at unblinded dossiers. Each half could then meet separately to decide which candidates they think should be interviewed. That is, if the department is planning on interviewing 20 candidates, then each half of the hiring committee would produce a ranked list of 20 candidates. Any applicant then appearing on both lists would definitely be interviewed.

What of students not appearing on both lists? There are a variety of ways of handling this problem. Here are a few:

(A) Suppose that exactly eight students appeared on both lists. There would then be 12 spots for interviews open. The committee could then decide to interview the top six members on the "unblinded half's" list that were not on both lists and the top six members on the "blinded half's" list that were not on both lists.

(B) In the case mentioned in (A), the members of the "blinded half" might then be given unblinded dossiers for the 24 applicants not on both lists. They would then be allowed a reasonable amount of time to review these unblinded dossiers. Finally, the committee would reconvene to hash it out over who should be selected for the remaining twelve interview slots.

There are, of course, other ways to deal with such a situation. But it seems to me that the basic proposal (splitting the committee in half, giving half blinded and the other half unblinded dossiers, etc.) would help to lessen the impact of pedigree bias while retaining any useful information contained by unblinded dossiers but not by blinded ones.

[Doug Portmore]

Greg,

I like your proposal in theory. I worry, though, about splitting the dossiers between two groups of reviewers. If different members of the hiring committee have different priorities and differing views on how to assess which candidates would best fulfill those priorities, then a candidate's chances for success might largely depend on which half of the hiring committee his or her dossier gets arbitrarily assigned to. I've been on hiring committees where a candidate ranked at the top of one's committee member's list was at the bottom of another's. At non-Leiteriffic universities, there is often a pretty diverse group of individuals with different priorities. Some value an analytic approach to philosophy and some disvalue that kind of approach. Some prioritize research over teaching, others vice versa. Some have a preference for someone who works in an area such as feminist philosophy, the philosophy of sexuality, or the philosophy of race and others don't. Since the majority of hiring decisions are made at non-Leiteriffic departments, I think that we should pay close attention to how things operate at such universities.

[Doug Portmore]

Here are a couple further thoughts about the worry that useful information such as "This is the best candidate to come out of Princeton in the last 5 years" will be lost if letters are blinded. First, we may wonder whether this is more useful than the candidate's publication record. Shouldn't the best candidate to come out of Princeton in the last five years have an easy time publishing in the best philosophy journals? And if they don't and tried to publish, then don't we have reason to doubt the truth of the claim. Perhaps, this is not always true, for, as Mark Schroeder reports here, he tried hard to publish as a graduate student but failed completely. But I suspect that this is an aberration. And, perhaps, if most schools started requesting blinded dossiers, then they would do a better job of not just training their students to be great philosophical thinkers but also training their students to write publishable papers. To succeed in publishing, one has to not only have good ideas but also know how to package them in just the right way. Perhaps, graduate programs need to focus more on teaching students how to package ideas so that they have a good chance of getting published. Second, I would like to point out that the best kind of letter, to my mind, is one that explains the candidate's research project, what contribution it makes, and why that contribution is new and significant. This kind of letter is more useful than one that contains claims such as "This student is the next Ted Sider" or "This student is the best to come out of Princeton in the last 5 years" without any arguments or evidence to back up those claims. Unless the letter writer supports such claims by giving a certain kind of evidence for them (the kind of non-purely-testimonial evidence that doesn't get lost when a dossier is blinded), the reader has to take the claims with a grain of salt unless the reader has had the experience of reading a number of letters from this letter writer over the years and knows whether or not he or she is prone to empty praise. And I suspect that in the case of the vast majority of hiring decisions many members of the hiring committee don't have this sort of information. Thus I worry that it's not all that useful for the vast majority of hiring committee members.

[Dale Miller]

A lot of the discussion here has centered on how much information pedigree gives you about how successful someone will be as a researcher. I wonder if it isn't as much or more important in terms of the information it gives about likely success in the classroom. Someone's writing samples and publications likely don't tell you much about how well they really know the subjects they list as AOCs, and you may be relying on them to teach these subjects. Knowing whom they studied with would help you to judge how ready they are to do so. Obviously this matters more if you're at a school with excellent undergraduates. Also, there is the question of how well someone's teaching experience as a TA will translate to your environment. How similar are the students whom they have succeeded with to yours? This is a consideration for a school like mine, which isn't terribly selective about whom it admits. People who did their Ph.D.s at Ivys will probably have, at a minimum, a difficult adjustment during their first few terms of working with our students. If I were at an Ivy, I'd be even more worried about someone making the transition from teaching mediocre students to teaching elite ones.

[Greg]

Doug,

I think your worry is based upon a misunderstanding of my proposal. On my proposal, the members of both halves of the committee would receive dossiers for every candidate. It's just that the members of one half would receive unblinded dossiers and the members of the other would receive blinded dossiers. No candidate's dossier would be "arbitrarily assigned to" either half of the committee.

[John Turri]

I'm persuaded that blinded CVs, writing samples and teaching evaluations are feasible and desirable.

People will find ways to communicate pedigree if they think it advantages them, but it won't be nearly as effective. And since the attempt would be patently transparent, it could even backfire.

If SCs began their work by generating an initial ranking based on blinded CVs, and then made adjustments based on a careful inspection of the (depending on the institution) blinded writing samples and/or teaching evaluations , it would make it more difficult (not impossible) for pedigree bias to creep in afterward. ("Oh, so this person went from 150 to 3 on your list after we found out she got her degree from [Elite U]? How very interesting!")

I think it would also go a long way toward making people more alert to pedigree bias, in themselves and others as the discussion proceeds. That's very important, because bias is often unconscious, and even well meaning people succumb to it.

Blinded letters sound good in principle, but they have most of the drawbacks already mentioned. More importantly, people just aren't going to go through the trouble. Greg's half/half proposal is interesting, but would involve even more work, and so would never happen either.

So I'm all for championing the cause of blinded CVs, writing samples, and teaching evaluation summaries. Those are feasible changes that could immediately make the hiring process more fair (and in ways that probably transcend pedigree bias).

[Greg]

Let me flesh out my proposal a bit more, since I'm pretty sure that Doug misunderstood it and I think John may have as well (since I don't think that his worry that it would involve more work and thus would never happen is justified).

Step 1: Hiring U publishes job ad requesting both unblinded and blinded dossiers (including blinded CVs, blinded writing samples, blinded evaluations, and blinded letters) from each applicant.
Step 2: Hiring U receives blinded and unblinded dossiers from n applicants.
Step 3: Blinded dossiers are divided from unblinded dossiers.
Step 4: Half of the hiring committee is given blinded dossiers; the other half are given unblinded dossiers.
Step 5: Members of the hiring committee look over dossiers (perhaps after each half meets separately to narrow down the number of applications to seriously consider).
Step 6: Each half meets separately to generate a ranked list of o applicants (where o is the number of applicants Hiring U would like to interview).
Step 7: The lists are compared. p candidates appear on both lists and are definitely interviewed.
Step 8: Which applicants will fill the remaining o-p interview slots are determined. How this is done might vary. The simplest suggestion is this. Interview the (o-p)/2 applicants that appeared at the top of the "unblinded half's" list but not on both and the (o-p)/2 applicants that appeared at the top of the "blinded half's" list both not on both.

This procedure would require a bit of extra work:
a. The inclusion of a few extra words in the job ad.
b. Each applicant preparing both blinded and unblinded versions of his/her dossier (minus letters).
c. Each letter writer preparing both blinded and unblinded versions of his/her letter.
d. Dividing blinded from unblinded dossiers.
e. Comparing ranked lists from the two halves of the hiring committee to find out who is on both.
f. Determining who receives the remaining interview slots.

(a) is clearly minimal work. (d) would probably not be too difficult either, esp. if it is specified that the blinded materials are to be paper-clipped together and the unblinded materials are to be paper-clipped together.

(b) and (c) may require a bit more work on the part of applicants and letter writers. But not, I think, much more. I know I wouldn't find it difficult to blind my dossier and I suspect that most letter writers aren't constantly dropping references to their institution or their colleagues in their letters, so it wouldn't be much more work for them either.

It would be pretty simple for an unaided human to do (e) and (f), but they could even be done by a simple computer program.

As far as I can tell, then, my proposal would require a little more work, but not so much more that it would never happen.

[anon]

I'm a "late bloomer", which is to say that I never quite fulfilled my potential until entering a PhD program. Since grade school the label "underachiever" has, deservedly, been mine. There are a number of factors. One is ADD, which I have only recently gained the maturity to control by changing my habits (waking up very early helps) and only recently decided to medicate. Despite very strong logic skills, my GPA suffered as an undergrad because I simply would not do the work in classes that did not grab hold of my interest. So, I went into a terminal MA program where I did merely ok. I got attention as promising with a "high pass" on the logic comp in my second semester, but also let a number of my professors down by sometimes turning in sub-par work, the product of procrastination. I am, by the way, currently thriving and receiving exemplary feedback on my work due to the changes I mentioned.

What does this all have to do with pedigree? Well, on the one hand I'm a beneficiary of pedigree. I'm currently in a middle tier graduate program, possibly due in part to the letter of recommendation I received from a well-known faculty member in the MA program I was in who was impressed with my work. On the other hand, students from top programs may have an advantage over me in the job market. I'm inclined to think that that is how it should be, however. At one time I beat myself up (a lot) over my academic shortcomings, viewing them as resulting from an unvirtuous lack of discipline. My current perspective is more clinical, but I still feel that those that have excelled throughout their entire academic careers have earned their advantages. That is, I'm more determined than bitter at the prospect of playing catch-up.

[Mark van Roojen]

I think you are going to have a really hard time convincing search committees to ignore information. So I would suggest a better way to acheive some of the spirit of the original, somewhat more egalitarian, proposal is to think about how to get committees to take into account additional information.

On the first point, Jamie and others have suggested several reasons why committees want to know who is writing a reference letter. It is worth putting those reasons in the context of real world searches where people have a limited amount of time. Letters have a larger role to play early in the process rather than later. I know I use various tricks to figure out which files to focus on. One of those is when one writer is writing for multiple applicants. Often you can order the applicants letters from that one writer. This has nothing to do with pedigree though it is information you would lose if you took out the info needed to determine pedigree. If you tell me I can't do that I am going to take a lot longer to get to the stage where I know which papers I'm going to spend my time reading. And if that takes longer I will be reading fewer papers. That's just one example. My main point is that when people are spending at most 15 minutes with a file (if that) as they make the first cut, committees are not going to do without these heuristics.

Myself I try to use additional information to find the files from less well-known departments that might be genuinely worth looking at. Obviously acceptances at good journals or conferences will be information of this sort. Letters from people at other programs are another. A good description of the project either on the CV or by the advisor that makes it sound like a really good idea is a third.

These aren't going to cancel out information about where a person went to school. But they might allow people from programs with less strong reputations make it into the group where we actually read their stuff more carefully. And in my experience, at that stage the intrinsic qualities of the sample somewhat swamp other data. My suggestion is that there might be other information of this sort that we should be paying attention to.

[John Turri]

Greg,

I understood you to mean roughly that, and I still do think people will be reluctant to do it.

a. The job ad is no big deal -- you're right.
b. I get the sense that an average job seeker puts in around 40 to 50 apps. Some put in many more. And it's a daunting and expensive task as it stands. Imagine doubling it -- twice the copies and twice the postage. It could easily end up costing a grad student an extra $500+. Maybe they should be willing to fork out the extra $$, though I wonder whether many could really afford it. So in effect you might be limiting the number of jobs they're able to apply for.
c. Many letter writers can't even get one copy of their letter done on time.
d & f. No big deal -- right again.

Regarding e (and step 6), it sounds like you're effectively doubling the amount of work a search committee does. Usually they'll divide n dossiers among, say, six people. Now you're dividing 2n dossiers among six people.

I hear SC members complain about how they don't have enough time to properly consider applications as it is. Imagine what would happen if you doubled their work load. (I'm not judging their complain, just reporting it.)

[John Turri]

And yes, I know it might seem that my own more limited proposal suffers from drawback b, too. But I would suggest asking for only blinded CVs, writing samples, and teaching evaluations. If the committee at a later point needs unblinded ones, those can later be solicited by email from a select group.

[Robert Johnson]

Much of this discussion concerns the conditional "If pedigree influences hiring, then the profession should try to neutralize that influence." Suppose pedigree does influence hiring (surely it does). Suppose further it does so mostly through bias -- not because hiring committees look at pedigree as a reliable indicator of quality or a calibration device for reading letters. It's still an open question whether that's a bad thing. I'm not sure I see evidence that it is bad. What are some of the bad things happening that can be attributed to pedigree bias? I can think of a few things one would expect to see if there was something nefarious going on, lots of people not getting tenure, for instance. But that isn't happening, as far as I can tell. Lots of people believe of themselves that their first job was not nearly as good as they expected it to be--indeed, it is true of many a philosopher I know. But that seems to cut across pedigree, and, anyway, everyone thinks she's above average. Lots of people also say they've been on hiring committees that didn't make an offer to the best candidate. But it's hard work judging potential and lots of room for error that has nothing to do with pedigree bias. Unless there's something going seriously haywire that could be pretty convincingly laid at the feet of pedigree bias, it doesn't seem to be worth trying to eradicate even if it does exist.

[Clayton Littlejohn]

Robert asked an important question:

What are some of the bad things happening that can be attributed to pedigree bias?

Here are three things that come to mind.

The first are the fairness considerations. It seems unfair for candidates with higher pedigree to get interviews for very attractive jobs when candidates with less pedigree but more pubs and conferences don't get the interview. I take it that there is evidence that this happens, but I don't know if you take this as also constituting evidence that bad things are happening.

Second, competition is good. If people coming from graduate school know they won't be competitive for attractive jobs because of where they do their graduate work, this will be bad for their development and bad also for those from prestigious departments who will be led to think that good jobs are rewards for completing graduate work. (Now, you might say that this gives undergraduates incentive to try harder to get into prestigious programs. It will. But, undergraduates aren't persons. They aren't responsible agents until 25. Also, it seems that an undergraduate's ability to get into a good graduate program can depend to some extent on where they received their undergraduate degree. I'm sure we don't want hiring decisions and decisions on who to interview to depend on some high school kid's ability to select an undergrad program that is a good launching pad to prestigious philosophy graduate programs.)

The third has to do with the signal it sends prospective graduate students. Let's say that pedigree is measured by ranking on the Gourmet report. That's a reflection of the overall strength of a department. Suppose I'm a graduate student interested in epistemology. Suppose that some of the better departments to do epistemology in have middling, low, or no ranking on the Gourmet report. I don't think it's a good idea for these prospective graduate students in my position to pick higher overall ranking and more prestige at the expense of the opportunity to work with better epistemologists and more epistemologists. If a hiring committee wants an epistemologist and had to choose between a candidate from a prestigious department that does little epistemology and a candidate from a middle, low, or no Leiter ranking that is a good place for doing epistemology, it seems sort silly to go for prestige in this instance (This assumes that prestige is understood in the way I've suggested.)

I'm sure there are other reasons, but I think these are good enough reasons to think bad things can be attributed to pedigree bias.

[anonymousse]

"Suppose further it does so mostly through bias -- not because hiring committees look at pedigree as a reliable indicator of quality or a calibration device for reading letters."

This way of putting things may suggest a mistaken view of what qualifies as bias. SCs could take pedigree into account, genuinely think pedigree is an indicator of quality, talent, etc., be acting in good faith and still be biased. They'd be biased if they were mistaken about whether pedigree was a good indicator or if they systematically overestimated (or underestimated) its value relative to other information.

Most who worry that pedigree biases hiring decisions are (or are most charitably taken as being) worried that SCs place too much weight on pedigree relative to things like publication record.

[Doug Portmore]

I like John's proposal if I understand it correctly. I think that, at first, search committees should look only at blinded CVs, writing samples, and teaching evaluations -- and, if needed, blinded versions of dissertation abstracts, statements of teaching philosophy, and such. After settling on a group of semi-finalists, the search committee would then ask these semi-finalists to send letters of recommendation and unblinded versions of their CVs and teaching evaluations -- and, perhaps, also transcripts. The search committee could, then, use this new information to settle on the list of finalists to be interviewed. This would go a long way toward alleviating various biases (such as those based on gender and pedigree). (I think that we need to worry about gender bias at least as much as we need to worry about pedigree bias.) And it would seem to involve little to no extra work for candidates, letter writers, and search committees.

Greg: I apologize for misunderstanding your proposal. My bad. I now favor John's proposal over yours for the reasons that John cites.

Robert: I would like to hear your response to Clayton and anonymous with whom I agree.

Dale: John's proposal would, I take it, alleviate this particular concern.

Mark: What do you think of John's proposal? Can't you just rely on blinded CVs and teaching evaluations to make the first cut? Once you have the group of semi-finalists, you can use the letters to infer the rankings of candidates coming out of the same program.

[Doug Portmore]

Here is some other useful information about avoiding bias.

[Robert Johnson]

Why is it unfair for candidates from "better" pedigrees to get interviews over those with "lesser" pedigrees, even given publication and conference paper differences? Is there some metric of 'fairness' here? Are the latter are better bets for tenure? Is there evidence of this?

Perhaps competition is good. Perhaps not. Not sure on that one, for philosophy at least.

Should we be trying to send signals to prospective grad students? Maybe we should just tell students about how the world works now in philosophy and let them decide for themselves what they want to do.

[Doug Portmore]

Robert,

Could you clarify your position. I thought that you were, for the sake of argument, conceding that there is a pedigree bias and were then wondering whether any bad things happen as a result of this bias. Now isn't a bias just a preference or inclination that inhibits the fair and impartial treatment of others. And so many would respond that it's bad in that in results in people being treating unfairly, which is bad. But now you're wondering whether the bias is unfair? But a bias is, as I understand it, inherently unfair. So your latest comment leads me to believe that what you mean by 'bias' isn't what I mean by 'bias'. So what do you mean by 'bias'?

[John Turri]

Hello Robert,

Why is it unfair for candidates from "better" pedigrees to get interviews over those with "lesser" pedigrees, even given publication and conference paper differences?

Clayton can of course speak for himself, but here's how I think of it.

Suppose you try to talk to me, but I ignore you because you're from Missouri and someone from Elite U is also trying to talk to me. That's unfair. And it's insulting. What a jerk I'd be. And that's just a conversation.

People applying for jobs are asking, at least in the first place, for an interview -- in effect, to talk to the search committee.

Suppose the search committee ignores you because you're from Non-elite U and someone from Elite U is also trying to talk to them. That's unfair. And It's insulting. And it's potentially harmful to you, in a way far beyond the (imaginary) harm caused by my earlier (imaginary) disrespect for your request for a conversation.

It'd be doubly unfair if both (a) you had already done some of the most important things that will be expected of you as a professional, whereas (b) the person from Elite U hadn't.

Anyway, for what it's worth, that's how I see things.

[Mark van Roojen]

Doug,
You wrote:
Mark: What do you think of John's proposal? Can't you just rely on blinded CVs and teaching evaluations to make the first cut? Once you have the group of semi-finalists, you can use the letters to infer the rankings of candidates coming out of the same program.

Once I've made the first cut I'm reading papers and the letters have much less of a role to play, but more importantly my reaction is that the information this proposal gives me does not allow me to make a first cut I would have any confidence in.

I've never figured out how to read teaching dossier's so that they tell me anything useful about the candidate's ability as a teacher or philosopher, at least not in the time I have to look at a file early in the process. Evaluations may be cherry picked or they may not. So comparing them is comparing apples to oranges at best. The one thing I can find useful is something that gives me a sense of the applicant's thought processes - sometimes a syllabus they've prepared or maybe an explanation of how they'd teach something. But trying to find that (and verify it is the candidates own thought) in each of 200 files takes more time than I've got when I have two weeks to narrow it down to thirty or fewer. And it tells me little about what I really want to know - are they decent at philosophy and do they have some good ideas they are working on.

The CV is the same. The presence of something on it can signal that I need to pay attention to this file, but the absence of anything at all is not any evidence of philosophical inability. At least half of the philosophers I most admire, many of them here, would have had very little on their CVs to hint at their potential right out of grad school. If you had blinded my CV when I went on the market there was nothing on it you would care to know. And there are much better philosophers than I on this list of whom that is also true.

So the proposal asks me to make the first cut on the basis of evidence I find unhelpful.

This is why I think a more reasonable approach to the percieved problem that pedigree is playing an undue role is to think about what you could add to a file to help people respond to something other than pedigree.

[John Turri]

Mark,

Just to be clear, my proposal included inspecting a blinded writing sample too for making an initial list, if desirable. (I mention this because it was not reflected in Doug's question, to which you directly responded.)

You said,

The one thing I can find useful is something that gives me a sense of the applicant's thought processes.

Why not just look at the writing samples, then? What better way to get a sense of someone's thought processes and how they do philosophy? (My guess: too time-consuming!) Or why not require a short (2pp) research statement, and focus closely on that?

Earlier you suggested something that I think is actually unfair to students from elite schools. You said,

I try to use additional information to find the files from less well-known departments that might be genuinely worth looking at.

I know you offer this in the spirit of trying to, as it were, "even things out" to some extent. But it does still seem unfair (and time-consuming, no?). Maybe the best we can do is to create multiple unfairnesses that aim to cancel one another. And perhaps students from elite places won't complain if the upshot is only that it "might allow people from programs with less strong reputations to make it into the group where we actually read their stuff more carefully."

[Simon Rippon]

Doug - at the top of the thread you said:

I often wonder how big a role pedigree plays in hiring decisions. I honestly don’t know ... it might be helpful to consider a hiring procedure that would, I think, totally eliminate pedigree ... as a factor ... I’m not advocating this procedure; I’m just employing it as a device for better understanding how pedigree might affect hiring decisions.

Yet in your recent comment you said (my emphasis):

I think that, at first, search committees should look only at blinded CVs, writing samples, and teaching evaluations ... This would go a long way toward alleviating various biases

This worries me, since during the discussion you've apparently decided that there is a pedigree bias that needs alleviating based on no evidence. Nobody should, of course, be complacent about the possibility of unfair bias. But neither should anyone simply assume without evidence that it exists. And one needs to make sure that the solution proposed will address the source of the unfair inequality that has been uncovered, rather than merely reducing the availability of relevant information.

One piece of relevant information, it seems to me, is that if a candidate's CV says "NYU" on the Ph.D. line, they have already beaten out many of their peers in a very competitive grad school entrance process, and at least demonstrated great promise to a bunch of brilliant philosophers in doing so. They will also, all things being equal, have had the opportunity to take advantage of an excellent education while in grad school that may or may not be matched elsewhere. Of course that doesn't entitle them to just walk into a job at the other end, but it's hardly irrelevant information when assessing their ability and promise as a professional philosopher (unlike gender, for example).

Moreover, I suspect that the "blinding" solution being proposed might have a very large but very temporary effect on changing how much relation pedigree has to hiring decisions. It is widely understood, now, that pedigree offers some advantage in making the first cut in the application process. (That's one of the many reasons why the grad school admissions process is competitive in the first place.) That being so, as Ralph pointed out in the earlier thread, many students at top programs are not encouraged to publish, present at conferences, etc., or are even discouraged from doing so. And given the widely understood difference that pedigree makes, they may well be perfectly rational to make the decision not to do these things until after they have finished their dissertations. On the other hand, there is already great pressure on students at lower ranked programs to publish and make presentations in order to distinguish themselves in the job market. So you can expect some pretty jam-packed CVs from these students. Blinding everyone's CVs without first warning everyone about changing expectations would have the effect of temporarily advantaging the latter group.

Why do I think the advantage may only be temporary? Because as expectations are adjusted, everyone will be competing for the same thing - publications in places as prestigious as possible. And students at higher-ranked programs tend to have a bunch of advantages that will make it easier for them to achieve this, all things being equal: better preparation before grad school, better financial support, lower teaching loads, better students, better colleagues to discuss things with, famous professors who frequently publish in prestigious places themselves, and so forth. Blinding CVs would do nothing to redress any of these inequalities.

John - It would be rude and insulting to ignore someone in a conversation merely because they lack any experience in teaching philosophy, yet it would not be unfair and insulting of a search committee to fail to grant an interview merely because the candidate lacked any experience in teaching philosophy (even where it is not announced as a strict requirement for the job). The norms for being granted a (casual) conversation are quite different from those for being granted a job interview.
Of course, one would hope that a hiring committee bases its decisions on the fairest possible assessment of the total evidence, and I think it would be very bad for a committee to rule out someone merely because they did not come from a Leiteriffic department. Whether that fact ought to play some role is a very different question.

[John Turri]

Simon,

I didn't understand your response to me in the last paragraph.

If it's true that an NYU education confers all the benefits you say, it'll manifest itself in a blinded dossier (esp. the writing sample). And if it does, then that person deserves (indeed, has earned) the interview, fair and square.

[Mark van Roojen]

John,

You write:

Earlier you suggested something that I think is actually unfair to students from elite schools. You said,

I try to use additional information to find the files from less well-known departments that might be genuinely worth looking at.

I know you offer this in the spirit of trying to, as it were, "even things out" to some extent. But it does still seem unfair (and time-consuming, no?).

I haven't actually talked about fairness here. I actually think that if the issue is overall fairness we must actually be thinking about the economic system as a whole and whether society ought to be structured so that people have roughly equal chances of similarly attractive jobs or some such issue. And then we have to worry about what we mean by equal chances, equal chances for equal talents, equal chances to get equal talents, etc., etc.,

Within that set of issues whether my credentials at a given stage helped or hindered me in getting a philosophy job seems small potatoes. I got my chance to go to grad school and I would have found perfectly good alternate employment if the philosophy thing had not worked out for me. That's easy for me to say now, of course. But I surely can't see that someone with my background and the chances I got could complain of unfairness if at a later stage my credentials were treated less respectfully than someone else's. I think there can be issues of fairness if certain sorts of considerations or causes lead to someone's being disadvantaged, like being in a discriminated against group, or whatever, but that was never a real possibility in my case. Gender is one of these categories and blinding might help with this where gender is concerned. So I should admit that I think there is some reason to favor blinding on that score, despite what I regard as the weighty reasons against it -- but I don't think coming from a less well-regarded department is like gender. The upshot here is that I think that if your main worries are about fairness, the most pressing issues occur much earlier on in the decision-making that effect a person's chances at education and employment.

So I'm thinking of the issue in terms of making the field open to real talent. My point was to try to bring some realism about how searches actually work into the discussion because it is important to distinguish the roles different bits of evidence play. If having a decent publications or having the strongest letter from a leading light in the relevant area each get you into the pile where people are reading your writing sample, then there is no disadvantage to those in the better-ranked departments, even if the people from those department will have a better chance at a letter from a leading light.

Getting into a decent graduate school and impressing the faculty is some evidence of some talent. So is getting paper in a good refereed journal or accepted at a good conference. Making an impression on someone not at your home institution is yet another bit of such evidence. Having someone explain the dissertation project/research program in such a way that it looks promising is another thing that gives such evidence. These are the sorts of things that you can make a rough assessment of in 5 to 10 minutes with a file. Readers who use all of these as criteria for being worthy of further attention will be picking up on more of the good files than readers who use only some. This is not to say they won't miss some good files. I just don't think that eliminating evidence at the early stages is going to make the process more reliable.

I think there is one relevant caveat. I do think that given the evidence we have about how people view gender, there can be biases introduced from having that information and blinding would get rid of that. But it would be very hard to make a file actually not indicate a candidate's gender. So I don't think this decides the blinding issue.

Is there another sort of evidence that in a relatively quick sort through files you think people should be looking out for?

(And pardon my spelling; each time I hit post I see another spelling error.)

[Greg]

I'd like to briefly respond to John's concerns about my proposal.

(a), (d), & (f): John agrees with me that these wouldn't constitute much more work.

(b): I agree with John that the added cost might be prohibitive. There is an easy fix, however. In particular, any department implementing my proposal could allow electronic applications, at least of those materials (CV, writing sample, etc.) for which the applicant is directly responsible. This would actually reduce the costs of applying. Perhaps letters could be sent by snail mail so as to ensure their confidentiality and not to further burden letter writers. The overall effect, however, would still be a reduction in costs.

(c): I don't think that having letter writers produce both a blinded and unblinded version of their letters would significantly increase the amount of time it takes for them to write them. They would have to remove any information indicating pedigree. But how much of this information is contained in a standard letter anyway? The name of the institution might be on the letterhead, but that just requires printing one version of the letter on standard paper. The letter writer would include his/her name in the closing, but that would simply need to be removed from the blinded letter. Besides this, however, it seems unlikely to me that letters standardly contain pedigree-indicating information.

(e): I take it that John's worry here is this. Often a hiring committee will divide the applications received among the members of the committee, so that if m applications are received and there are n members of the committee, each member will be responsible for m/n applications.

This is a serious worry for my proposal. However, it's seriousness depends (at least in part) on how common this procedure is. Doug mentioned cases earlier in which a candidate who appears at the top of one committee member's list appears near the bottom of another's. Presumably this sort of situation wouldn't occur unless both committee members were responsible for the same application. Also, I'm under the impression that in at least some committees, things are not done in this way. Instead, when all applications are received, the committee meets to cull down the applications to a manageable number and then each committee member independently evaluates each of the remaining applications before finally meeting to determine who should receive interviews. In that case, something similar could be done on my proposal. The only difference would be that each half of the committee would meet separately to cull down the number.

Of course, this all depends on the inner workings of the committee. A committee with the type of procedure John describes would have more work on my proposal, perhaps too much more to make it feasible. On the other hand, a committee with the type of procedure I just described would not have more work. In any case, I don't claim that every department concerned with pedigree bias ought to adopt my procedure. I just claim that some departments thus concerned can adopt that procedure without incurring significantly more work.

[Simon Rippon]

John:

[an advantageous education] will manifest itself in a blinded dossier (esp. the writing sample). And if it does, then that person deserves (indeed, has earned) the interview, fair and square.

I think you're begging the question about desert here. I gave some reasons earlier to think that blinded dossiers may not accurately reflect either talent or hard work.
I didn't mention writing samples, but let me put the worry this way: given the arbitrariness that we all know exists in journal refereeing, do you seriously think that search committees are in a position to reliably rank a couple of hundred papers that may well be outside every SC member's AOS? If not, they will need to rely heavily on other information as well.

[John Turri]

Simon,

I don't know what questions I might have begged, though I'd be happy if you explained which ones.

As for your analogy with anonymous, unaccountable referees -- provided search committees take their work seriously, no search committee member would be in a similar position, so I'm not impressed (yet) by the comparison.

And please don't forget that I'm not advocating eliminating or blinding letters. On my proposal those still come into the picture, but only later.

[John Turri]

Thanks, Mark. I continue to appreciate your injections of realism about how searches work.

I agree that there are larger issues of fairness. Yet there's still a narrow but important issue of fairness here too.

I know you said you don't like eliminating evidence in an early stage. But doesn't not reading writing samples in the early stage pretty much have the same effect as eliminating evidence at that stage?

Just so I'm clear on this: is it your view that the various bits of evidence you mentioned, which can be ascertained in 10 minutes or less with a file, provide a more reliable way of initially sorting files than actually reading the writing sample and/or research summary? Or is it that that's as reliable as it's going to get, within the time limit people are willing to spend on a file?

P.S. I agree with you one-hundred-percent about the importance of overcoming gender bias. I had this in mind earlier when I mentioned helping to eliminate biases other than pedigree.

[David Sobel]

A successful case that we ought to blind ourselves to pedigree (or some of ourselves) would have to make a case that there are other, more reliable ways to assess talent which are not much more difficult to determine than reading unblinded letters. Reading all the papers from hundreds of applicants is just not a serious alternative unless there are going to be course reductions for people on search committees. So the argument hinges, it seems to me, on the ease and usefulness of people’s publication records in assessing philosophical excellence (I’m setting aside teaching numbers and evaluations as the latter raise similar issues).

Without speaking to the comparative question of the excellence of information between unblinded letters and publication record, let me say a few things that worry me about relying even more on publication record. It takes a long time to get a paper accepted, often more than a year. It seems to me that either 1) grad students will only have 1 or 2 shots at getting something published before going on the market and the lottery of referees is too random to make accurate assessments of people based on only 1 or 2 trials by journal or 2) grad students will reorient their behavior so as to give themselves more shots at publications before going on the market. This means sending off more, less good papers, and coming up with self-standing papers that are suitable for publication which likely means responding to literature rather than trying to come up with a new approach and sending these papers to less prestigious journals so as to have a better chance at having some positive news on the publication front.

I wonder if those who like making applications more pedigree-free agree that their case hinges on the excellence of the info we get from publication records? If so, I wonder if they disagree with me about the above reasons to worry about the excellence of such information (or embrace the changes that 2 above suggests)?

[Doug Portmore]

David:

Thanks for getting to the crux of the matter and raising some powerful objections that warrant equally powerful responses. I’m not sure that I’m up to the task, but let me give it a go.

Let me start by agreeing that the case for the Turri Proposal, as I’ll call it, hinges on whether its method of selecting candidates is likely to be a more reliable way of identifying the best scholars than the current method is. And now I’ll try to respond to your worries.

First, I think that you underestimate the number of times graduate students would be able to submit. You think that it’s along the lines of one to two times at most. I think that it’s more like four to five times on average. I started submitting in my third year, and submitted at least six times (and I think seven) before completing my Ph.D. I submitted only two of my best papers (the two that won awards in my department). Of course, it helped that many times the paper was summarily rejected without even being sent on to referees.

Second, it’s important to realize that a sizeable portion of the job candidates for any position are not fresh out of graduate school. Some are people who spent the last six years at some Leiteriffic school and were denied tenure. Many are people who for some reason or another are not happy with the tenure-track job that they now have. They have a partner that lives on the opposite coast. They don’t like the city that they’re in. Or, perhaps, they don’t like their teaching load. Many are candidates who have been in temporary positions for the last one or more years. Many of these candidates have proven publication records. They have not been vetted just by two to four referees. Many of them have four or more publications and some of them may even be at places like Phil. Review and Ethics. Some of these people have proven that they can publish in the best venues and that they can be sufficiently productive even under the worst of conditions. And if the Turri Proposal was implemented, many of these candidates will include some of the best and brightest out of the Leiteriffic schools who for one reason or another didn’t publish at all while in graduate school and so had to work hard and publish while teaching in temporary (perhaps, even adjunct) positions their first few years out of graduate school.

And, yes, I think that if search committees went with blinded CVs rather than unblinded letters and CVs as a means of identifying a list of semifinals this would prove a more reliable method of identifying the best researchers. They would be choosing from a list of semi-finalists who had all proven their ability to do the very thing that they need to do to receive tenure: publish. These people have proved not just that they can elicit glowing letters from their teachers, who have a vested interest in their success; they have proven themselves to the editors, editorial board members, and reviewers of some of the best journals all of whom have no vested interest in their success.

Admittedly, this means that some very talented philosophers who just didn’t acquire the knack for packaging their brilliant ideas in neat little publishable bundles will be forced to resort to teaching in adjunct or visiting professor positions for their first year or so out of graduate school. And they will have to learn how to publish under conditions that are less than ideal: conditions where they’re teaching many service courses and applying to jobs every year. But this is already true. There are many very talented philosophers who have to prove themselves on the battlefield, so to speak, before they get a shot at a tenure-track job. I think that this proposal is likely to produce a fairer playing field and I think that it will also help search committees make better choices concerning the first round of cuts. I think that people have a tendency to overvalue the unproven potential superstar and to undervalue the proven excellent but non-superstar philosopher.

Now, you also worry about what effects this pressure to publish as a graduate student will have on the discipline. Let me point out that, as far as I can tell, all but those at the most Leiteriffic schools already feel the pressure to publish as a graduate student, so I don’t think this proposal calls for any significant shift in the practices of the vast majority of graduate students. Also, I don’t think, as you suggest, that this proposal will result in graduate students “sending off more, less good papers, and coming up with self-standing papers that are suitable for publication which likely means responding to literature rather than trying to come up with a new approach and sending these papers to less prestigious journals so as to have a better chance at having some positive news on the publication front.” I advise and was advised to focus of trying to publish my best paper (or, perhaps, my best two papers), to work on them long and hard before submitting them, and to submit them to the top journals. Submitting them to mediocre journals only proves that you’re mediocre, and that’s the last thing that you want to prove. I also wouldn’t advise graduate students to focus solely or mainly on trying to publish critical pieces. I think that this could doom their career. Everyone knows that it’s much easier to pick apart someone else’s argument than to come up with a self-standing argument of your own. And search committees want to see that you can do the latter. So that’s the kind of article that you should try to publish. So, (1) I don’t think that most graduate students would reorient their behavior, as most already assume that they must publish to fare well on the job market, and (2) I think that those who do need to reorient their behavior because they didn’t already feel the need to publish as a graduate student will not, if properly advised, do so in the detrimental way that you suggest.

[Doug Portmore]

Simon,

you've apparently decided that there is a pedigree bias that needs alleviating based on no evidence.

Not true. I have a fair bit of evidence that there is a pedigree bias. Some of this evidence comes from self-reflection and the fact that I find this sort of bias in myself. Some of this evidence comes from my experience on search committees—evidence that I’m not willing to share in this open forum. And some of this evidence comes from talking to people who hired me and learning from them that my coming out of UCSB was a problem for them even though I had been out of graduate school for more than eight years. Of course, this evidence is not the best sort of evidence, but it’s the best evidence that I have. There is, I think, better, more systematic evidence of gender bias in academia. And this is one very important bias that I hope to in part alleviate by blinding CVs.

One piece of relevant information, it seems to me, is that if a candidate's CV says "NYU" on the Ph.D. line, they have already beaten out many of their peers in a very competitive grad school entrance process, and at least demonstrated great promise to a bunch of brilliant philosophers in doing so. They will also, all things being equal, have had the opportunity to take advantage of an excellent education while in grad school that may or may not be matched elsewhere.

But shouldn’t this translate into publications insofar as it’s relevant to their professional success? I know more than a handful of really smart, talented, and well-educated people who for one reason or another fail to publish much or at all. One’s job as a professional philosopher is not just to be brilliant and talented. An essential component of one’s job is to translate one’s talent into tangible results: i.e., publications.

Blinding everyone's CVs without first warning everyone about changing expectations would have the effect of temporarily advantaging the latter group.

Don’t worry: they’ll have plenty of warning so as to avoid such a temporary advantage. This sort of change, if it occurs at all, isn’t going to happen overnight.

Why do I think the advantage may only be temporary? Because as expectations are adjusted, everyone will be competing for the same thing - publications in places as prestigious as possible.

This is what I want: everyone competing for publications in places as prestigious as possible.

Blinding CVs would do nothing to redress any of these inequalities.

I’m not trying to address all inequalities by blinding CVs. That would be silly. To address some of the inequalities that you speak of, we would need to make adjustments to our society’s educational system and not just to the method by which search committees settle on lists of semi-finalists.

[David Sobel]

You make interesting points in reply. I do think this is at least one heart of the issue and so I'd be keen to hear what others thought about it. Thanks for your thoughtful reply which I need to think about.