Just something to think about.  Interesting story in the ABA Journal from a few weeks ago. Law partners were asked to review a legal brief and analyze the writing.  Even though all participants read the exact same legal brief, they were given different information about the supposed author:

Sixty partners from 22 law firms who agreed to participate in a “writing analysis study” received copies of the memo. Half were told the memo was written by an African-American man named Thomas Meyer, and half were told the writer was a Caucasian man named Thomas Meyer. Fifty-three partners completed the task. Of those, 29 received the memo supposedly by a white man and 24 received the memo supposedly by a black man.

The reviewers gave the memo supposedly written by a white man a rating of 4.1 out of 5, while they gave the memo supposedly written by a black man a rating of 3.2 out of 5. The white Thomas Meyer was praised for his potential and good analytical skills, while the black Thomas Meyer was criticized as average at best and needing a lot of work.

Reviewers found an average of 2.9 out of seven spelling and grammar errors in the memo by the white Thomas Meyer and 5.8 out of seven errors in the memo by the African-American Thomas Meyer….

Here’s the ABA article, and here’s the original report the ABA was referencing.  Several commentators on the ABA website noted problems with the study (for example, we do not know what other factors might have affected the partners’ evaluations).  The consulting firm that conducted the study does attempt to limit the scope of its results:

In order to create a study where we could control for enough variables to truly see the impact of confirmation bias, we did not study the potential variances that can be caused due to the intersection of race/ethnicity, gender, generational differences and other such salient identities. Thus, our conclusion is limited to the impact of confirmation bias in the evaluation of African American men in comparison to Caucasian men. We do not know (although we plan to study the issue in the very near future!) how this impact will splinter or strengthen when gender and/or other identities are introduced.

I’ll keep an eye out for that follow-up study.  Meanwhile, the firm suggests countering inherent bias by taking proactive steps to limit subjective and subconscious influence to the extent possible, as in this example:

In one law firm where we found that minority summer associates were consistently being evaluated more negatively than their majority counterparts, we created an interruption mechanism to infuse the subjective with objective. We worked with the firm to create an Assignment Committee, comprised of 3 partners through whom certain assignments were distributed to the summer associates and through whom the summer associates submitted work back to the partners who needed the work done. When the work was evaluated, the partners evaluating the work did not know which associate had completed the work. The assignments for this process were chosen judiciously, and there was a lot of work done to ensure buy-in from all partners. At the end of the summer, every associate had at least 2 assignments that had been graded blindly. The firm then examined how the blind evaluations compared with the rest of the associate’s evaluations and found that the blind evaluations were generally more positive for minorities and women and less positive for majority men.

Advertisements