“Citation Stickiness” A Study by Bennardo and Chew

Their paper “Citation Stickiness,” to be published in The Journal of Appellate Practice and Process, offers an extensive look at a core assumption in the legal field: that briefs by both parties are pivotal in every case. Their research raises questions about the quality of attorneys’ briefs, whether courts disregard parties’ filed documents and how effectively law schools teach legal research.

Bennardo and Chew teach in UNC’s Research, Reasoning, Writing and Advocacy Program. In classes, “we talk about how important it is for students’ credibility to help judges by finding and citing the best cases,” Chew says.

But as a law clerk, she adds, “I found judges didn’t always rely heavily on the cases lawyers cited. Law clerks often started the research over.”

With growing interest in using empirical research to look more closely at judicial decisions, Bennardo and Chew studied 325 cases in federal appellate courts, with the aim of contributing data and a new methodology for more discourse. The opinions they analyzed had 7,552 unique case citations, and the briefs in the 325 cases had 23,479 unique case citations

“Citation stickiness,” a phrase Bennardo and Chew coined, refers to how often a citation in a brief appears in the court’s opinion. A “sticky” citation appears in a brief and then also in the court’s written decision. By contrast, an “unsticky” citation doesn’t get referenced in the opinion. Other citations, which weren’t mentioned by either party but appeared in the judicial opinion, probably came from the court’s own research.

The study’s key finding was that 49 percent of cases cited in courts’ opinions were cited in a brief by at least one party; the remaining 51 percent came from another source.

Regarding what the ideal percentage of parties’ citations in judicial decisions is, “We don’t have an opinion,” Chew says. “The study has sparked discussion about what the right amount should be. We hope our study helps other researchers figure out some of the ‘whys.’”

The article notes Bennardo and Chew were surprised by the key results. “One would hope parties’ attorneys and courts discuss the same case law. We found there’s not as much overlap between the cases they’re discussing as most people might hope,” Bennardo says.

Their paper has generated discussion in various forums, from Twitter to a piece in the Legal Skills Prof Blog to a podcast on legal scholarship. Some research platforms may use the study’s methodology, and attorneys could apply the calculation to their own briefs to see if their citations are sticky.

The article doesn’t make recommendations. “We’re thinking through whether to suggest changes to how people teach or practice,” Chew says.

She and Bennardo may study characteristics of attorneys who write briefs to look for reasons for different stickiness rates. “That’s one of the open questions,” Bennardo says, “and none of the judge characteristics we looked at — including political affiliation, age and judicial experience — answered it.”

The feedback is instructive.

“I’ve enjoyed discussions about citation stickiness with other legal writing and research professors,” Chew says. “These are the things we’re trying to teach students how to do.”