Forty points: how psychopathy is measured, and who holds the pen

The experts were paid to do what they did every working week: read an offender's case file and score it on two standard risk instruments. There were 108 of them, forensic psychologists and psychiatrists. The files were the same for everyone. One thing differed. Some had been led to believe they were consulting for the prosecution. The rest believed they were consulting for the defence (Murrie et al., 2013).

Those who thought they worked for the prosecution gave the offenders higher risk scores. Those who thought they worked for the defence gave the same offenders lower ones. The differences were not trivial: effect sizes reached d = 0.85, a large effect by the usual conventions. Daniel Murrie and colleagues called it an allegiance effect (Murrie et al., 2013).

One of the instruments was the Hare Psychopathy Checklist–Revised, the PCL-R. It is one of the most widely used measures of psychopathy, and it is offered in court in decisions as heavy as civil commitment and capital sentencing (Boccaccini et al., 2008; DeMatteo et al., 2020). To understand what a score on it means, it helps to know where it came from and what it actually counts.

From a book of case histories to a checklist

In 1941 the American psychiatrist Hervey Cleckley published The Mask of Sanity, a book of detailed case histories of patients who presented as charming, intelligent and untroubled, and who lived lives of repeated, purposeless harm with no apparent remorse (Cleckley, 1941). Cleckley's portrait was clinical and literary. It was not a measurement.

Robert Hare turned the portrait into a research instrument. In 1980 he published a checklist for assessing psychopathy in prison populations (Hare, 1980), which became the PCL-R. The current manual sets out 20 items: among them glibness and superficial charm, a grandiose sense of self-worth, pathological lying, conning and manipulation, lack of remorse or guilt, shallow affect, callousness and lack of empathy, impulsivity, poor behavioural controls, juvenile delinquency and criminal versatility. A trained evaluator scores each item 0, 1 or 2 using a semi-structured interview and a review of the person's records. The total runs from 0 to 40 (Hare, 2003).

Read the list again and notice how many items are judgements about a person's manner. "Superficial charm" and "shallow affect" are not facts sitting in a file. Someone decides whether they are present, partly present, or absent, and that someone is a human being sitting across from another human being in a particular role.

The evaluator effect

Murrie's experiment was controlled. Marcus Boccaccini, Darrel Turner and Murrie had earlier looked at what happens in the field. They gathered PCL-R scores for 321 sex offenders assessed by 20 different state-contracted evaluators, as part of screening for civil commitment as sexually violent predators (Boccaccini et al., 2008).

More than 30% of the variation in scores was attributable to differences between evaluators. The mean scores given by two of the most prolific evaluators differed by almost 10 points. For 22 offenders scored on two or more occasions, agreement between evaluators was low, with an intraclass correlation of .47 (Boccaccini et al., 2008).

Ten points is a quarter of the whole scale. In research on the PCL-R, a total of 30 is commonly used as a threshold for the label "psychopath". An offender can cross that line, or not, depending on who is assigned to the case.

None of this means evaluators are dishonest. In the allegiance study the experts were simply misled about who had hired them, and the effect still appeared. It is what happens when an ambiguous, high-judgement task is done by people who know which side they are on. The instrument is reliable in research conditions, with trained raters scoring independently. The courtroom is not research conditions.

A measure is not the thing it measures

A deeper problem sits under the scoring. Jennifer Skeem and David Cooke warned in 2010 that the field was in danger of treating the PCL-R as if it were psychopathy itself. Factor analysis of the checklist's items, done without a theory of what psychopathy is, cannot reveal what psychopathy is. And because several items record criminal history, a score can rise because a person has offended a lot, which makes the link between "psychopathy" and offending partly circular (Skeem & Cooke, 2010).

Their reading of the evidence was that criminal behaviour is better understood as something that goes along with psychopathy, not one of its defining parts (Skeem & Cooke, 2010). That distinction sounds academic. It is not. If a measure partly counts past crimes, and is then used to predict future crimes, part of what it is doing is restating the record in clinical language.

How well does it predict?

Min Yang, Stephen Wong and Jeremy Coid compared nine commonly used violence risk tools in a meta-analysis of 28 reports. All nine predicted violence at about the same moderate level. The exception was Factor 1 of the PCL-R, the interpersonal and affective traits most people think of as the core of psychopathy, which predicted violence among men at chance level. The authors' advice was blunt: the tools are essentially interchangeable for predicting violence, and their accuracy is not high enough for them to be used alone for decisions such as preventive detention (Yang et al., 2010).

In 2020, thirteen researchers, among them some of the most published names in PCL-R research, published a joint statement on its use in US capital sentencing, where it had been offered as evidence that a defendant would be violent in prison. Their position was that the evidence does not support using it that way (DeMatteo et al., 2020).

Psychopathy outside prison

Psychopathic traits are not confined to people with criminal records. Delroy Paulhus and Kevin Williams described the "Dark Triad" in 2002: narcissism, Machiavellianism and psychopathy, three overlapping but distinct sets of traits measurable in ordinary samples of people (Paulhus & Williams, 2002). Most people with high scores on these measures are not offenders. The traits are dimensions on which everyone sits somewhere, not a category that some people belong to.

That matters for how the word "psychopath" is used. It turns a position on several continuous scales into a kind of person. Labels like that shape how everyone afterwards reads the person's behaviour, a process sociologists have studied for a long time, and one that horror fiction acts out: how societies decide who the monster is.

What a score is

The PCL-R is a serious instrument with a large research base. It identifies a cluster of traits that is real and that matters. The research on it, much of it by the people closest to it, also says plainly what it is: a structured clinical judgement, made by a person, from an interview and a file.

Read that way, a score carries three questions with it. Who scored it, and in what role? How much of it comes from the criminal record the decision-maker already has? And is it being used for something it was shown to do? The 108 experts in the allegiance study were skilled, qualified and acting in good faith. Their numbers still moved with the side that hired them (Murrie et al., 2013). The number is only as neutral as the conditions it was made in.

Short answers

What is the PCL-R?

The Hare Psychopathy Checklist–Revised is a 20-item rating scale. A trained evaluator scores each item 0, 1 or 2 from a semi-structured interview and a review of records, giving a total from 0 to 40 (Hare, 2003).

Is a PCL-R score objective?

It is a structured judgement, not a blood test. In one experiment, forensic experts who believed they had been hired by the prosecution gave higher risk scores than experts who believed they had been hired by the defence, for the same files (Murrie et al., 2013). In real cases, a large share of score differences came from which evaluator did the scoring (Boccaccini et al., 2008).

Does the PCL-R predict violence?

About as well as other risk tools, which is moderately. A meta-analysis found nine common tools predicted violence at about the same level, and the interpersonal and affective factor of the PCL-R predicted violence among men at chance (Yang et al., 2010).

References

1. Boccaccini, M. T., Turner, D. B., & Murrie, D. C. (2008). Do some evaluators report consistently higher or lower PCL-R scores than others? Findings from a statewide sample of sexually violent predator evaluations. Psychology, Public Policy, and Law, 14(4), 262–283. doi.org/10.1037/a0014523
2. Cleckley, H. (1941). The mask of sanity: An attempt to reinterpret the so-called psychopathic personality. Mosby.
3. DeMatteo, D., Hart, S. D., Heilbrun, K., Boccaccini, M. T., Cunningham, M. D., Douglas, K. S., Dvoskin, J. A., Edens, J. F., Guy, L. S., Murrie, D. C., Otto, R. K., Packer, I. K., & Reidy, T. J. (2020). Statement of concerned experts on the use of the Hare Psychopathy Checklist–Revised in capital sentencing to assess risk for institutional violence. Psychology, Public Policy, and Law, 26(2), 133–144. doi.org/10.1037/law0000223
4. Hare, R. D. (1980). A research scale for the assessment of psychopathy in criminal populations. Personality and Individual Differences, 1(2), 111–119. doi.org/10.1016/0191-8869(80)90028-8
5. Hare, R. D. (2003). Manual for the Hare Psychopathy Checklist–Revised (2nd ed.). Multi-Health Systems.
6. Murrie, D. C., Boccaccini, M. T., Guarnera, L. A., & Rufino, K. A. (2013). Are forensic experts biased by the side that retained them? Psychological Science, 24(10), 1889–1897. doi.org/10.1177/0956797613481812
7. Paulhus, D. L., & Williams, K. M. (2002). The Dark Triad of personality: Narcissism, Machiavellianism, and psychopathy. Journal of Research in Personality, 36(6), 556–563. doi.org/10.1016/S0092-6566(02)00505-6
8. Skeem, J. L., & Cooke, D. J. (2010). Is criminal behavior a central component of psychopathy? Conceptual directions for resolving the debate. Psychological Assessment, 22(2), 433–445. doi.org/10.1037/a0008512
9. Yang, M., Wong, S. C. P., & Coid, J. (2010). The efficacy of violence prediction: A meta-analytic comparison of nine risk assessment tools. Psychological Bulletin, 136(5), 740–767. doi.org/10.1037/a0020473