The Cost of Judicial Citation Style: An Empirical Investigation of Citation Practices in the Federal Appellate Courts

 

Casey R. Fronk†

(WORK IN PROGRESS – LAST UPDATED 06/29/09)

   Since the early 1960s, computerized legal research technology has enabled judges and their law clerks to access legal information quickly and comprehensively. Particularly for appellate judges, who rely on wide-ranging legal research when writing opinions, this technological change has had special resonance. This Article attempts to quantify the effects of computer-assisted legal research on the federal judiciary by empirically analyzing citation patterns over the past fifty years. The results of this analysis demonstrate that computerized legal research has had statistically significant effects on the amount and style of citation in judicial opinions. Although the average number of cases cited in opinions has doubled between 1957 and 2007, the number of cases cited only in string citations has decreased by nearly the same percentage. This Article suggests that such results can be explained by a basic economic theory of judicial citation in which judges respond to the decreasing cost of opinion production by discarding string citation for more effective communicative techniques.

 

Introduction

Recently, commentators have identified an emerging empirical trend in legal scholarship that focuses on “testable hypotheses and large data sets.” [1] The vast majority of this scholarship has concentrated on the content, rather than the style, of judicial opinions. Some empirical analyses have attempted to correlate judicial voting patterns with party affiliation (normally using the party of the appointing president as a proxy).[2] Others have uncovered statistically significant connections between a judge’s gender, race, and prior experience and that judge’s voting tendencies in a variety of cases.[3] Nearly all, however, have employed a binary method of coding judicial votes that fails to expose less palpable avenues for the influence of judicial ideology. Indeed, few have devised reliable empirical methods for investigating the effects of environmental or political features on the literary style of judicial opinions.[4] 

   Citation analysis, which has usually been exploited to investigate judicial influence and prestige, is a fertile method for examining these more subtle aspects of judicial opinions. Perhaps because interdisciplinary citation research has generally focused on citations externally, as evidence of quality or influence, many early judicial citation analyses explored the correlation between the number of times a legal scholar or judge was cited and the reputation of that scholar or judge.[5] Other early legal citation analyses were simply descriptive, reporting information about citation practices in a variety of contexts.[6] It is only more recently that citation analysis was aimed internally to find evidence of biases in the citation practices of judges.[7] These studies, however, find only minute correspondence between citation style and ideological valence; and measure such effects in a temporal vacuum. More importantly, these empirical investigations fail to account for other (perhaps more significant) effects on the citation behavior of judges—such as the basic cost of citation.[8] 

   Likewise, while some commentators have theorized that judicial citation practices have changed dramatically over time,[9] few recent studies have utilized the large data sets or sophisticated statistical techniques necessary to confirm expansive hypotheses about the shifting nature of citation technique. In some cases, these limitations were temporally imposed: the early citation studies that appeared in the 1970s and 1980s, for example, could not adequately analyze the effects of computer-assisted legal research on judicial opinions. In other cases, technological restraints limited feasibility: several later analyses, for example, explicitly aimed to isolate the effect of computer databases on citation practices;[10] but none had the requisite database size to draw statistically significant conclusions about the widely-suspected correlation. It is only recently, after the introduction of more advanced case analyses by the online commercial legal databases, that a systematic study of judicial citation practices has become methodologically feasible.

   This Article exploits these novel methodological options in an attempt to fill the existing empirical gap. Specifically, this Article analyzes the quantitative and stylistic change in citation practices in the federal appellate courts over the past fifty years, finding that judicial citation in these courts has changed dramatically since 1957. This analysis utilizes three datasets: first, a collection of 1200 U.S. Circuit Courts of Appeals opinions from 1957 to 2007 in ten-year intervals; second, a compilation of 688 cases written by two of the longest-serving federal appellate judges; and finally, a subset of 600 federal appellate cases containing a single majority and dissenting opinion, subdivided based on the relationship between the political ideologies of the majority and dissenting judges. The second dataset, which inspects the citation habits of two individual judges, is employed primarily to buttress the first: specifically, to demonstrate that the results found in the main dataset are not primarily influenced by the changing composition of the federal judiciary. Accordingly, the results of the analysis can be seen to approximate the effect of the dramatic change in the cost of producing judicial opinions created by the onset of computer-assisted legal research. The third dataset analyzes the extent to which citation practices can be explained by a theory of communicative citation: specifically, whether a dissenting judge is more likely to cite widely where he does not share the political ideology of the majority judge and whether the cost of legal research influences the extent of that citation.

   For several reasons, an analysis of federal appellate decisions is likely to be more productive than an investigation of state court or federal district court opinions over the same time period. First, although there have been some variations in the type of cases generally adjudicated before federal appellate courts in the last fifty years,[11] the types of cases considered by federal appellate courts are not nearly as varied as those cases adjudicated by the state courts. Second, data on the ideological bearing of each federal judge (coded as the party of the appointing president) is more readily available than information of the same sort for state judges, some of whom are elected.[12] Third, for purposes of developing a random sample, using federal appellate courts rather than federal district courts ensures that the stylistic vagaries of a single district court will not overwhelm the random sample.[13] Finally, the smaller sample size produced by searching only federal appellate courts creates a more manageable grouping for research purposes.[14] Notably, despite these advantages, this Article appears to be the very first citation study of its kind to focus on federal appellate opinions: similar previous analyses have generally concentrated on state-level[15] judicial opinions.

   The resulting analysis produced several remarkable conclusions. In contrast to earlier studies, which hypothesized that the use of string cites would increase as computer-assisted research techniques and massive legal databases reduced the costs of search, this study indicates that the opposite has occurred: rather than simply increasing the amount of string citations, federal appellate judges have instead expanded their qualitative analysis of cited cases—leading to significantly longer opinions with more meticulous discussion of each case cited. In empirical terms, the average percentage of cases cited only in string citations has declined from slightly above 24 percent in 1957 opinions to slightly above 8 percent in 2007 opinions.[16] Likewise, the percentage of cases cited expositorily[17] over the same period has increased from approximately 9 percent in 1957 opinions to 23 percent in 2007 opinions.[18] More remarkably, most of the decrease in the percentage of unique string-cited cases and much of the increase in the amount of cases cited expositorily occurs between 1977 and 1987, which parallels the proliferation of cost-effective computer-assisted legal research in law firms and courts.[19] Specifically, the average percentage of unique string-cited cases in 1987 is only slightly over half that of 1977, and the average percentage of cases cited expositorily nearly doubles over the same decade. This result is also borne out in the opinions of two individual judges over the same period,[20] indicating that the decline in string citation is not attributable to a changing judicial workforce.

   Also, the data indicate that judicial citation practices in dissenting opinions are correlated to a certain degree with the ideological affiliation of the majority opinion writer—and more importantly that these practices have changed dramatically based on the decreasing cost of legal research. In fact, among the 600 cases analyzed, the percentage of unique cases cited in “hostile” dissents (where the party of the appointing president of the majority and dissenting judges is opposite) is almost double the same percentage in non-hostile dissenting opinions.[21] Moreover, there are some significant temporal differences within the datasets: in both 1977 and 1987, for example, the percentage of unique cases cited in hostile and non-hostile dissents was nearly identical—suggesting perhaps that the technological advancements in computerized legal research reduced the communicative value of dissent citation.

   In general, this empirical analysis of judicial citation shows that the citation style of judicial opinions can be conceived in simple microeconomic terms. As the costs producing a judicial opinion decline, judges will be able to produce more both on a quantitative and qualitative level. More specifically, the style of citation appears to depend intimately on its contemporary technological format: when computer-assisted searches of cases returned only case names, and not full text opinions, judges were more likely to include such citations in string-citation format. A comparison between the concurring and dissenting datasets displays the staggering extent of this effect: in essence, the economic cost of citations is significantly more determinative of citation style than the political valence of the judges on the panel issuing the opinion—perhaps suggesting that while judicial ideology is determinative of the content of judicial opinions, the style of the opinion is generally immune to such influences.

   This Article proceeds in five parts. The first Part discusses the technological transformation beginning in the late 1950s that considerably reduced the cost of writing federal appellate opinions, and hypothesizes about possible effects on citation styles over that period. Parts II and III present evidence for these microeconomic hypotheses—analyzing data from an aggregate sample of federal courts of appeals cases between 1957 and 2007 and from two specific samples of appellate opinions by long-serving circuit court judges to argue that the cost of an opinion is largely determinative of its style. Part IV then investigates the effects of this changing cost structure on the citation content of dissenting opinions to conclude that the rapid decline in the costs of citations in the 1970s and 1980s drastically altered the typical dispersal between citations in ideological and non-ideological dissents. Finally, Part V concludes.

 

I. The Historical Experiment: Computerized Research and The Cost of Citation

 

   As early observers recognized, the mid twentieth–century advent of computer-assisted legal research substantially altered the cost structure of the legal research process. In 1960, reflecting on the future of electronic data retrieval systems, one practitioner predicted that “the day will surely come when the judge [can] devote a greater part of his working time to the professional function of analyzing and applying the law, and much less of it to the slow, laborious and expensive process of gathering the materials.”[22] Others envisaged that when practitioners “are freed from some of the demands of routine and drudgery in the performance of their functions, they may have greater opportunity and impulse to devote themselves to the effort of intellectual creation necessary to keep the substance of the law as modern as its techniques promise to become.”[23] All understood the necessity of such systems in organizing the vastly increasing precedential output of courts: “Perhaps when lawyers begin to be swamped by a deluge of literature, they will come to a keener appreciation of the efforts being made currently . . . to devise methods for performing some parts of the research process automatically.”[24]

If, as these early commentators predicted, judicial opinions represent merely the product of a number of economic inputs, this revolutionary change in the cost of one input (citation) should have a measurable effect on the amount (or perhaps form) of judicial output. Matching the change in output to the historical development of electronic research devices, however, would be impossible without a general outline of the rate and type of technological advances most likely to affect judicial productivity. This Part, therefore, attempts to sketch the historical situation in which judicial production operated: Part I.A gives a general historical overview of the rise of computer-assisted legal research; Part I.B. focuses more specifically at the judicial recognition and use of these alternative production methods; and Part I.C. outlines a microeconomic theory of judicial citation practices that may explain some of the historical variation in citation style.

 

A.   The Historical Experiment

  

   Although it is tempting to characterize the advance of computer-assisted legal research as an instantaneous change in the cost structure of opinion production—in which even the earliest computers “could scan the legal wisdom of the ages in instants, green glyphs etching precedents across the screen to suit any need;”[25] the actual story is one of slow progression from print to digital searching. Primarily due to physical constraints, the first computer research systems were only able to query a small universe of statutory materials manually preprogrammed into an accompanying database.[26] In these early manifestations, programmers converted each relevant statute into punch-card form and then entered these into the computer’s database, which compiled the information to create an alphabetical list of every statutory word.[27] In order to locate relevant materials, a legal researcher would input a list of words (or combinations of words), and the computer would print a list of statutes containing those words.[28] The print-out would contain the names of the statutes and the number referent of each term searched, and could also reproduce the full-text of each statute.[29] Of course, these early systems had a number of significant drawbacks: they were “wholly incapable, without specific instructions, of dealing with language difficulties such as those posed by ambiguities or synonyms;”[30] were severely limited in searching scope;[31] and were extremely slow[32] and expensive.[33]

   Due to these limitations, it was not until the mid-1970s, nearly fifteen years after the first computerized research systems were introduced, that computer-assisted legal research became commercialized nationally.[34] LEXIS was introduced in April of 1973, and was marketed primarily to major New York law firms.[35] Westlaw followed shortly thereafter, and acquired its first subscriber in April of 1975.[36] These early commercial efforts were both limited and expensive for subscribers. LEXIS, for example, originally had a minimum use cost of $36,000 per year,[37] and did not provide access to a large volume of information. Although the system provided content such as the U.S. Code, Supreme Court cases, and certain federal appellate and district cases in specialized legal fields,[38] by the late 1970s only the cases of eleven states were available for research.[39] Likewise, for several years Westlaw provided researchers with only the West headnotes[40] rather than the full text of cases—it was not until as late as 1979 that Westlaw offered a substantial full-text database.[41] Even though the LEXIS minimum charge was reduced to $18,000 in 1975 and then was waived in 1976, moderate usage during that time period was estimated to cost between $20,000 and $25,000 per year.[42]

   Because of the expenses and technical complexities of the commercial computer research systems, widespread commercial usage of LEXIS and Westlaw did not appear until nearly a decade later. The LEXIS system in the late 1970s and early 1980s required the use of a dedicated computer terminal connected via telephone line to the LEXIS main computer in Dayton, Ohio.[43] When using the system, the user would connect from his terminal to the LEXIS mainframe to conduct the searches—the results of which would be displayed on the office terminal.[44] Training attorneys and paralegals on the technical complexities of these terminals was rather expensive: Westlaw estimated the cost of user education in the early 1980s at “approximately $100 per day”—and around $2700 total to train an average-sized law firm.[45] Law firms’ typical searches were also quite costly. In 1976, using the LEXIS system required expenditures of around $100 per hour—primarily due to LEXIS’ costs of inputting and storing its vast conglomeration of legal information[46]—and the searches were often rather slow.[47] Even by 1983, the estimated cost of such computerized queries for a law firm was between $2 and $4 a minute.[48] Accordingly, most law firms were slow to adopt the service. LEXIS, for example, had slightly less than two hundred subscribers in 1976.[49] Even by 1983, observers reported only intermittent use at law firms.[50] 

   Business usage increased substantially in the late-1980s after several technological improvements augmented search scope and efficiency on LEXIS and Westlaw. Both commercial services introduced a larger variety of search terms throughout the 1980s: for example, by 1986, both systems offered the universal character and the root expander to enable searches for partial words.[51] Additional new search queries permitted users to not only search a single library of cases, but the entire federal case database or the entire state law database. Users were also able to search these new massive databases topically for specific areas of law.[52] Both services also offered further infrastructure options: Westlaw began selling dedicated terminals, and LEXIS enabled access through non-proprietary computers (as Westlaw had for several years).[53] Subscribers responded positively to these search improvements. By 1988, analysts were predicting growth for each commercial service of between 20 and 30 percent a year;[54] by 1989, the one millionth LEXIS user identification code was issued; and by 1990, over 100,000 searches were conducted on LEXIS in a single day.[55]

   As has been noted elsewhere, much of the explosive growth in computerized legal research in the late 1980s and early 1990s can be attributed to the success of the commercial services’ aggressive grassroots marketing campaign. When LEXIS and Westlaw began to compete for users, each attempted to gain long-term subscribers by subsidizing the costs of student use.[56] Both bestowed free-access passwords on student users and provided law school libraries with subsidized computer equipment—beginning with terminals and continuing with dedicated printers and personal computers.[57] Accordingly, students began to use the online systems more often and more exclusively in law school and later, in law practice. As one commentator noted in 1994, “[f]or many American law students the only common research training experience they receive is the training provided by LEXIS and Westlaw, a training experience that will follow the student into practice if he or she works in a large law firm environment.”[58]  

   External and internal technological improvements in the 1990s further refined the commercial systems’ usability. One such advancement was the personal computer, which permitted LEXIS and Westlaw users to operate the electronic legal research program in tandem with a word processing program.[59] Researchers could copy and paste paragraphs of text from the research program into the word processing program—significantly decreasing the editorial demands of legal research.[60] Access to the Internet also increased general usability: in 1996, LEXIS subscribers were able to research online through the LexisNexis Office software suite.[61] Both LEXIS and Westlaw also began to provide more comprehensive user assistance. In the early 1990s, the commercial systems began offering twenty-four hour telephone assistance with trained researchers providing research tips and advice.[62] Later during that decade, LEXIS introduced pay-per-use access as an alternative to subscription service for smaller firms and solo practitioners.[63] These varied technological developments encouraged research growth. By 1998, LEXIS was processing over 600,000 searches and adding over twelve million documents to its online service every day.[64]

   Although this explosive growth did not continue unabated to the present day, several substantial additions to the online commercial databases after the turn of the century have significantly increased the scope of legal research. First, both systems began to publish a greater number of cases—including cases not published within the federal reporters. Second, Westlaw in particular began to make available a larger variety of legal materials, including briefs and other trial documents. [65] In fact, Westlaw recently enabled new queries that allow researchers to explore over 1.4 million federal and state briefs and 2.75 million trial documents.[66] Access to such briefs enables users to use prior research to find analogous cases more efficiently. Perhaps due to these new services, usage continued to increase after the 1990s. Now, each service processes hundreds of thousands of searches per day and adds over 1.2 million cases per year.[67]  

   In sum, the growth of computer-assisted legal research is best divided into three periods. The first period, between 1960 and around 1977, saw the introduction and (limited) use of primitive legal research systems as a viable alternative to manual research. The second period, between 1977 and around 1993, witnessed the proliferation of the commercial research systems and vast technological improvements in the structural architecture and interfaces of such systems. The final period, between 1993 and the present, exhibits slow but steady growth in both the type of content available on the commercial systems and the number of users, but not the same significant technological advancements of the second period. Thus, if the cost of research is reflected in legal citation practices, the imprint of these three eras of computerized legal research should be reflected in the style of judicial opinions written within those periods. 

 

B.       Computerized Research Systems in the Federal Courts

 

   Early in the development of computer-assisted legal research, federal judges recognized the time-saving function of digitalized precedent. In 1961, Judge John R. Brown of the Fifth Circuit wrote an article describing the advantages of computer research.[68] Presciently, Brown claimed that computers could solve the increasing problem of storing and accessing relevant precedent.[69] Although recognizing that the current print-focused organization system was innovative, Brown concluded that the system’s requisite rigidity “makes search a time-consuming and expensive proposition.”[70] Computers, he thought, would “save precious professional time in the routine low-order search which finally uncovers the few pieces calling for close study and lawyer-like judgment.”[71] 

   Occasionally, as in First National Bank of Birmingham, Alabama v. United States,[72] federal judges explicitly mentioned computerized research in their opinions. In First National Bank, for example, Judge Brown specially concurred in order to “focus professional attention on another effective use of the marvels of electronic machines in the vastly expanding business of litigation and its disposition.”[73] In that case, a question arose over whether the facts of the tax case under appeal consisted of “an isolated case of an isolated set of taxpayers in an isolated non-repetitive setting, or [whether it was] one of those test cases so often tenderly coveted by tax counsel, private and government.”[74] The Fifth Circuit utilized an IRS computerized data system to establish that “the appeal really involves no true question of estate tax law,” and thus could be decided on state rather than federal grounds.[75] Other early mentions were less grandiose. In Holcomb v. United States, for example, the Seventh Circuit noted the ineffectiveness of the LEXIS system in finding an unknown case citation.[76] 

   Despite these professed uses of computerized legal research, an early study of actual federal court research methods indicates that computer consultation was the exception rather than the norm before the late 1970s. A Federal Judicial Center study shows that between January 1 and May 31, 1976, use of computer-assisted research systems by federal appellate law clerks ranged from 0.26 to 7.33 hours per month.[77] Usage rates for district court clerks were even less—ranging from 0.00 to 3.15 hours per month.[78] For the appellate sample, these hours of use translates to an average of between 1.00 and 16.20 research sessions per month.[79] Although the caseload of circuit court judges was not at that time nearly as substantial as current levels,[80] this amount of computer-assisted research seems almost negligible by today’s standards. For example, assuming that law clerks spend only 10 percent of their work hours researching and that most law clerks endure a forty-hour work week (both rather conservative assumptions), today’s clerks would spend at least four hours a week researching—a fourfold increase from the 1970s average. (Interestingly. anecdotal evidence suggests that in the late 1970s, appellate clerks were significantly more likely to rely exclusively on appellate briefs as a source for the relevant research.[81]) Moreover, late-1970s usage levels between judges varied remarkably: on the D.C. Circuit, for example, clerks for one appellate judge were over twenty times more likely to employ computer-assisted legal research than the clerks of another appellate judge.[82] This variance suggests that many appellate judges and clerks had not yet adopted computerized research as a consistent research method.

   Access is an important factor explaining the computerized research usage levels of appellate courts in the late-1970s. At this time LEXIS, the more widely available commercial service, required a dedicated terminal in order to access its databases—and accordingly only judges “on-site” at the terminal’s location had access. Even after May of 1976, when “call-in centers” were installed in the Fifth and Ninth Circuits, enabling appellate judges to telephone from non-terminal locations to use the computerized research systems, appellate judges exhibited some reticence over taking advantage of their access to the terminals.[83] Between July and December of 1976, for example, Fifth Circuit judges utilized the call-in center only eighty-six total times: an average of only five times per judge, and only 0.8 times per judge per month.[84] Although the Ninth Circuit experiment was more successful, it too failed to encourage widespread adoption of computer-assisted legal research. Specifically, between June 1 and November 30, 1976, appellate judges in the circuit used the system a total of 170 times.[85] This translates into an average of around 10.6 uses per judge in the sample time period, or only 1.8 uses per judge per month. Anecdotally, these low usage numbers are probably best explained by judges’ lack of familiarity with the terminal system and the frequent technical glitches in early operation. At the time, one Ninth Circuit judge noted that the call-in system was “sometimes tied up for days which precludes our use of [the system] for short-term ‘rush’ projects.”[86] Another remarked that his “clerks simply have not developed the habit of [using the call-in system].”[87]

   A helpful proxy in determining the attitudes of the federal courts to computerized legal research is the judicial mention of such research as part of “reasonable attorney’s fees” under cost-shifting statutes. Beginning in the later 1970s, several courts began to note (and even laud) the use of computerized research as part of a reasonable fee recovery. In Pitchford Scientific Instruments Corp. v. PEPI, Inc., for example, the court noted that although “[d]efendants object to the charge for the use of Lexis computer service, describing it as ‘an impermissible anthromorphism’,” the cost was reasonable since “[t]his service . . . replaces by instantaneous and supposedly infallible retrieval, many hours which would be billable if performed by human talent.”[88] Likewise, the District Court of Rhode Island, in 1983, concluded that although it was “unable to find any authority directly in point anent the recoverability of computer charges for the use of computer-aided legal research systems, this Court believes that such charges should be recoverable in certain cases” since ruling otherwise “would be an open invitation to law firms to use high-priced attorney time to perform routine research tasks that can be accomplished quicker and more economically with Lexis.”[89] The Third Circuit, in an earlier case, declared that “use of computer-aided legal research such as LEXIS, or WESTLAW, or similar systems, is certainly reasonable, if not essential, in contemporary legal practice.”[90] Although such statements are almost nonexistent before 1977,[91] they are found in increasing quantity afterwards.[92]

   Thus, while computer-assisted legal research is unlikely, before at least the late 1970s, to have widely altered the research or citation practices of most judges on the federal appellate courts, evidence from early surveys indicate that judges had a remarkably positive attitude toward such systems and their further proliferation. In fact, judges in the late 1970s were not primarily concerned about the efficiency or reliability of computerized research systems but instead with their cost-effectiveness. Judges responding to a Federal Judicial Center inquiry about computer-assisted legal research nearly uniformly cited cost-effectiveness as the only remaining barrier to adoption. One judge found the computerized research system “generally to be a very effective efficient and easy system of legal research especially useful for certain types of work,” but worried, “I have no idea of the cost charged, however, and therefore cannot state that it is more economical than traditional ‘book’ research.”[93] Another judge responded that computer-assisted legal research “is extremely useful—but, in terms of cost, it doesn’t do the job of another clerk, or even half a clerk.”[94] Other judges, however, were less worried about the cost. One judge, for example, reported that computerized legal research was “a great time saver” and that his office “has come to rely on it substantially.”[95] 

   It was not until the early 1980s that judges would wholeheartedly adopt these electronic researching mechanisms.[96] By 1982, computerized research had proliferated to an extent that one federal court felt confident concluding, “The bulk and complexity [of legal precedent] have grown to such an extent that even experienced lawyers cannot function efficiently today without the support of special tools, such as the computer research systems of FLITE, JURIS, LEXIS and WESTLAW.”[97] By 1985, the commercial systems had become so integrated into the research patterns of the federal courts that one judge could use the phrase “sitting hunched over a LEXIS or WESTLAW terminal” as a metaphor for legal research.[98]

   Although the process was rather laborious, the ultimate result of integrating computer-assisted legal research into the judicial system was a significant reduction in the costs of research. While no studies have yet estimated the actual reduction in costs over the entire period between the early 1960s and the present brought about by the computerized research revolution, an early survey conducted by the Federal Judicial Center in federal appellate courts supports the traditional story hypothesizing substantial time savings created by computer-assisted legal research. In 1976, only three years after the commercial introduction of computerized legal research systems, appellate judges and their clerks estimated that the systems saved between 4.1 and 4.4 hours of legal research time per week of use.[99] Since these users also estimated that they spent between 21.6 and 25 hours per week conducting legal research before the introduction of the computerized systems,[100] even the very early research systems saved a substantial amount of time—around 17 to 19 percent of total research time. Once the computer systems became more common and more sophisticated, this reduction in research time would seem to produce substantial cost savings for federal courts—especially in the domain of judicial citation.[101]

 

C.       A Simple Microeconomic Theory of Citation

 

   The production of judicial citation is an especially labor-intensive task. Although in many cases the vast majority of this labor is borne by the appellant and appellee, who must provide thoroughly-researched briefs to the appellate court, a judge must also research separately in order to find strands of precedent the parties may have missed or to buttress a decision not directly aligned with the argumentation in the briefs. Especially in the era before computerized research systems, legal research was one of the most time-consuming aspects of a judge’s work: one early study estimates that over 30 percent of an appellate judge’s time was spent on finding and verifying applicable precedent.[102] 

   In this sense, appellate judges are simple consumers of citation, who are constrained in their consumption of opinion inputs by budgetary (primarily temporal) restraints. Specifically, each federal appellate judge has a fixed amount of wealth (time) with which he can purchase inputs for an opinion. Although with an unlimited income a judge would consume citation until the marginal value of the next citation is negative, physical constraints inhibit a judge’s ability to muster precedent flippantly. Thus, like consumers, judges must allocate limited resources among a variety of products—including, perhaps, court administration,[103] pro bono work,[104] and even moonlighting or leisure.[105] A change in judicial income should accordingly create interesting effects on judicial consumption, since a judge must reevaluate his consumption patterns in lieu of his newfound windfall.

   When the cost of legal research declines consumer theory predicts two potential effects: an income effect, and a substitution effect. The reduced cost of research essentially creates a positive shift in a judge’s budget constraint—he can now consume more of any number of goods (including leisure, for example).[106] This is the income effect, whereby a judge will consume more total citations (or perhaps more pro-bono work or leisure). Notably, the decreasing cost of legal research may also produce a substitution effect. Namely, as the cost of some types of legal citation decline relative to other types of legal citation, a judge will reallocate consumption between the two types of citation (or even more generally between two types of activities, such as opinion-writing and leisure).[107] 

   Moreover, increasing aggregate judicial purchasing power might have the effect of altering the relative demand for different types of citation. For example, if expository citation (in which a judge details the reasons for citing a case in paragraph form) becomes less expensive relative to string-citation (in which cases cited for a proposition are merely listed, without explanation), perhaps the demand for expository citation will rise, and the demand for string citation will fall. Depending on the levels of consumption after the budgetary constraint is lifted, it may be possible to characterize certain types of citation as either normal or inferior goods.[108] 

   This microeconomic theory of citation correlates nicely with the communicative theory of citation proposed by some courts and commentators. Although many alternative theories of judicial citation have been proposed,[109] this Article utilizes this “communicative” approach to citation practices—understanding citation (and opinion-writing generally) as an attempt by judges to signal to some audience both the basis and the amount of support for their decisions.[110] One motivation for judicial citation may be to communicate doctrinal necessity and thereby guarantee public confidence.[111] As the Supreme Court recently remarked in Rita v. United States,[112] “judicial decisions are reasoned decisions [and] [c]onfidence in a judge’s use of reason underlies the public’s trust in the judicial institution.”[113] Therefore, “[a] public statement of such reasons helps provide the public with the assurance that creates that trust.”[114] Less public-oriented communicative theories of citation, however, have also been proposed. For instance, some commentators have suggested that lower court judges use written opinions to indicate their suitability for promotion,[115] and others have, especially in the context of dissenting opinions, argued that citations are an indicator to future judges of the strength of alternative reasoning.[116] Whether judges are communicating to their peers, their superiors, or to the general public, however, this communication must have some inherent cost. Indeed, the cost of the communication is precisely what creates its informational value by differentiating among potential communicators.

 

II. Citation Practices in the Federal Appellate Courts,      1957–2007

  

   In order to investigate the technological effects on citation style and perhaps on citation signaling by judges, this Article conducts an empirical analysis on federal appellate cases from 1957 to 2007. Part II.A establishes the empirical methodology. The following part, II.B, surveys the increases in total citation over the past fifty years and correlates these significant increases to the technological milieu in which they occurred Then, Part II.C analyzes the incidence of string citation in opinions over the relevant time period, concluding that the changes in string citation can be directly correlated with the technological advancements detailed in Parts I.A and I.B. Finally, Part II.D investigates the broader stylistic variances that gave rise to the string citation effects, and concludes that these too can be explained predominantly through the microeconomic framework.

 

A.   Data and Methodology

 

   This empirical analysis uses data from 1200 federal courts of appeals cases decided in the fifty years between 1957 and 2007. Specifically, twelve sets of one hundred circuit court cases were gathered: 600 published[117] opinions in which one judge concurred and wrote a separate opinion,[118] and 600 published opinions in which one judge dissented and wrote separately.[119] These opinions were collected at ten-year intervals. Thus, the complete data set consisted of one hundred concurring and dissenting opinions for 1957, 1967, 1977, 1987, 1997, and 2007. In some cases, due to a dearth of concurring or dissenting federal circuit court opinions in the year allotted, additional opinions were compiled from the preceding year in order to create a consistent set of one hundred opinions.[120]

   Each set of opinions in the main data set was analyzed by counting the number of unique case citations within both the majority and concurring (or dissenting) opinion. This was accomplished by means of Westlaw’s “Table of Authorities” feature,[121] which compiles an exhaustive list of the cases cited in a judicial opinion. Even if a case is cited more than once in the opinion, the case appears only once in the Table of Authorities, and thus is only counted once in the final calculation. Notably, because multiple citations to a single case are not necessarily indicative of the amount of research used in creating an opinion, the resulting number of “unique” cases cited in an opinion probably better approximates the cost of the opinion than a measure of raw citations. In every dissenting opinion, but not in every concurring opinion, the number of unique dissenting case citations (those appearing only in the dissenting opinion, and not in the affiliated majority opinion) was also computed.[122] The final dataset of 1200 appellate opinions contained 27,490 case citations.

   For purposes of further analysis, the case citations in each relevant opinion were divided into four classes that approximate the analysis each case received in the opinion. This was accomplished by using Westlaw’s “depth-of-treatment” rating, which divides cited authorities into four categories.[123] The first category, “one-star” authorities, indicates cited authorities that are cited only in a string citation[124]—cases this Article classifies as “unique string-cited cases.” The second category, denoted by a depth-of-treatment rating of two stars, includes cases cited as the primary authority for a proposition in at least one point in the majority or concurring/dissenting opinions.[125] The three-star category includes cases cited several times within an opinion, usually accompanied by at least a paragraph of textual analysis in the opinion.[126] The final category, four-star cases, designates only those cases that are considered in a significant amount of depth—usually accompanied by several paragraphs of analysis when cited.[127]

   Finally, to determine the overall depth-of-citation percentages within each sample, the mean of the percentage of cases cited in each depth classification within the sample was calculated. There are a number of advantages to this percentage-based approach that are not present in a simple citation count. Since this is an attempt to measure the somewhat ephemeral notion of judicial “style,” merely counting the number of unique string-cited cases and calculating the mean number of such citations per opinion would not illuminate differences in the percentage of string citation, since the amount of cases cited in the mean opinion has escalated dramatically in the last fifty years.[128] Thus, while in some cases the mean number of unique string-cited cases may rise slightly in the later samples, this is not necessarily indicative of an increase in “junk” citations: instead, the increase in the total number of cited cases may overwhelm (at least in percentage terms) the increase in the raw number of string citations.

   The result of this analysis is to yield, perhaps for the first time, a quantifiable measure of judicial citation style. Previous citation studies have debated, more specifically in the context of the average age of cited opinions, whether a judge’s method of citation is more closely correlated to personal “taste” or external considerations.[129] The differences in the mean number of cases cited, the percentage of string citation, and the percentage of expository citation identified below support the external economic account of judicial citation. Namely, the results indicate that external factors—most distinctively the costs of legal research—have nearly determinative influences on most aspects of judicial citation style.

 

B.       Total Citation, 1957–2007

 

   Perhaps the most obvious indicator of external coercion on judicial citation style is the number of cases cited within an opinion. If, as is suggested by the history of computerized legal research, appellate judges are able to more cost-effectively research applicable precedent, their purchasing power has concomitantly increased. In effect, computer-assisted legal research allows judges to consume more utility-producing activities. Although, as Judge Posner’s judicial utility function suggests,[130] judges might allocate this increased purchasing power across a variety of products; including leisure, moonlighting, and other utility-generating activities, one important avenue for consumption is opinion-writing. Likewise, if judicial clerks are driving the citation process, their increased ability to consume citations should also have an important effect on the citation style of their judicial employer.

   It is not surprising, given the tremendous research advantages conferred by computerized research systems, to find that judicial consumption of citations has increased radically in the last five decades.  More specifically, there have been two remarkable modifications in the mean number of unique cases cited in the federal appellate opinions since 1957. First, in the decade between 1967 and 1977, the mean number of cases cited per opinion jumps substantially: from 17.85 in 1977 to 23.94 in 1977, an increase of over 6 cases (over 34 percent) per opinion. The second statistically significant increase occurs between 1997 and 2007, when the mean number of cases cited per opinion swells from 24.19 to 31.14—an increase of nearly seven cases (nearly 29 percent) per opinion.

 

 

 

   These findings buttress the results of previous citation studies estimating the mean number of cases cited over the same period. A study by Judge Posner in 1985, for example, found that the average federal appellate opinion in 1960 contained 12.4 citations, and that the average 1983 opinion contained 24.7 citations—both findings remarkably close to this Article’s results.[131] Whether these increases are due specifically to technological advances, however, is open to question. Prior studies of citation indicate that the mean number of cited cases per opinion began to increase dramatically beginning as early as the 1940s, perhaps because an increasing stock of precedent created more opportunities for citation, or because changes in judicial staffing levels increased opinion production capacity.[132] 

   Interestingly, this analysis did not uncover a significant discrepancy between the mean number of cases cited in concurring and dissenting opinions within any single sample year. Although in both 1977 and 2007, where the most significant growth in mean cases cited is observed, the measured difference between the mean cases cited in concurring and dissenting opinions is larger than in any other sampled year; these differences are not statistically significant. This suggests that the income effect observed in Table 1, in which judges consume more citations as their income increases, does not engender a significant substitution effect between raw citations in concurring and dissenting opinions. Instead, citation increases in both types of opinions, and judges allocate relatively similar amounts of research time to concurring and dissenting opinions before and after their purchasing power increases. 

   More importantly, however, the data in Table 1 suggest that a technological explanation cannot fully account for the observed increases in mean number of cited cases. For example, one major increase in total citation occurred between 1967 and 1977, before the widespread adoption of computerized legal research systems in the federal courts and private practice. Likewise, the second major increase occurs between the 1997 and 2007 samples, although no major computer-assisted research developments transpired during that decade.

   One alternative to the technological explanation deserves mention: some commentators have suggested that the increase in the number of available law clerks for each court of appeals judge is directly responsible for the recent increase in citation.[133] Historically, all federal circuit judges received one law clerk in 1930.[134] This number was doubled in 1970, when circuit judges were given a salary allowance that enabled them to hire more clerks.[135] Although circuit judges received enough money to hire three clerks since 1974, [136] this was not commonly done before 1980, when a congressional enactment that increased staff salary allowances enabled federal judges to hire three clerks at normal pay levels.[137] Shortly thereafter, in the early 1990s, judges were given the option of employing four clerks and one secretary instead of three clerks and two secretaries.[138] Notably, these changes in potential staffing levels only roughly correspond to Table 1’s findings: between 1967 and 1977, when potential clerk levels doubled, the total number of cited cases increased most dramatically (by slightly over 34 percent). The second major increase in mean total citation, however, fails to correspond to the increased staffing levels:  the increase from two to four clerks occurred before 1997, but the second major increase in case citations (around 29 percent) did not appear until after the 1997 sample.[139] Moreover, even if clerking levels may roughly approximate the changes in total citation, they are much less helpful in rationalizing the discrepancies in string citation over the same period.

 

C.       String Citation, 1957–2007

 

   Judges have often derided excessive string citation, characterizing the practice as needlessly justificatory and occasionally harmful. Judge Posner, of the Seventh Circuit, castigates law clerks’ writing style as making “an ostentatious display of the apparatus of legal scholarship—string citations, copious footnotes, abundant references to secondary literature.”[140] Judge Posner finds law clerk’s reliance on string citations “superfluous” and often “inaccurate.”[141] Other judges have railed against the practice in appellate briefs. According to Judge Harry Pregerson of the Ninth Circuit, for example, using string citation is “the third sin” of appellate brief writing, since such cites “are rarely useful or impressive.”[142] Despite (or perhaps in response to) these misgivings, however, the data shows that string citation has actually fallen out of favor in appellate opinions over the last fifty years—becoming nearly a novelty in recent years.

   Two measures of string citation are informative for temporally assessing the influence of economic factors on such citation. First, the percentage of cases in an opinion cited only within a string citation, and not further analyzed, might respond significantly to changes in the cost of legal research or legal writing: if the cost of either declines, judges would have more time to analyze each case in further depth and present their findings in expository fashion. Likewise, the average number of cases cited uniquely in string citations within an opinion generates a similar approximation of the costs of legal citation. Accordingly, both measures were analyzed to determine whether significant patterns arose over time.    

   The striking results of this analysis of unique string-cited opinions are found below in Table 2, which displays the number of string-cited cases in an average federal appellate opinion as a percentage of the total number of cases cited in the average opinion. Specifically, the first column reports, in ten-year increments, the mean percentage of unique string-cited cases in the total sample of opinions analyzed in each sample year. The two following columns display the same data for the relevant samples of concurring and dissenting opinions, and the final two columns show the same results for dissenting opinions in which the ideology of the majority opinion- (as measured by party of appointing President) was either the same or the opposite of the dissenting judge.

 

 

   At first glance, several interesting trends emerge. A comparison of the columns reporting string-citing concurring and dissenting opinions reveals that only minimal (and non-significant) differences exists between the mean percentage of unique string-cited cases in the concurring and dissenting sample opinions within the sample years. For example, the greatest difference between these two samples in a given year is only 2.9 percent, between the 1977 dissenting opinions and the 1977 concurring opinions. There is a dramatic difference, however, between the percentage of unique string-cited cases in 1957 and in 2007. In the case of concurring opinions, the mean percentage drops from nearly 25 percent in 1957 to below 10 percent in 2007; and in the dissenting opinion samples, the percentage drops nearly as radically, from over 23 percent in 1957 to slightly under 7 percent in 2007. Even more interesting is the nature of this change: in the first two decades considered in the sample, the total percentage of unique string-cited cases varies almost imperceptibly—indeed, the changes from 1957 to 1977 are not statistically significant. In the twenty-year period between 1977 and 1997, however, the mean percentage of string-cited cases drops precipitously; declining by over 15 percent in those years.

   The large standard deviations in each measured sample indicate, of course, that these results are not necessarily predictive of any single opinion within the sample. Although, as noted above, opinions from 1957 are generally distinguishable from contemporary opinions, the wide variance in individual cases creates some ambiguity.[143] These variations are most likely the result of two factors. First, a large number of opinions in each sample were simply not long enough to include string cites. In fact, 24 percent of the opinions in the 1957 concurring opinions sample, and 25 percent of the 2007 concurring opinions, did not employ string cites. These are predominately those opinions that are only a few pages long and contained only a few cases that were cited expositorily. Second, judges within the samples generally revealed divergent citation styles. Thus, while the sample as a whole demonstrated a general coherence, individual literary styles created large discrepancies within the individual opinions.[144] The macro-level changes, nevertheless, are remarkable.

 

 

Figure 1

Percentage of Unique String-Cited Cases

by Year and by Type of Opinion

 

   In particular, Figure 1’s graphical representation of the mean percentage of unique string-cited cases demonstrates the suddenness of the 1977–1997 decline. The mean percentage of unique string-cited cases is relatively constant both before and after the large drop in that twenty year period, suggesting that the cause of the drop was confined to a significant external change within those two decades. Notably, the decrease in the percentage of string citation directly parallels the increased proliferation of computerized research systems: there is no significant change in string citation until the 1980s, after computer-assisted legal research was widely adopted, and the decrease tapers off after 1997 when new advancements in computer-assisted legal research and word processing were not as revolutionary.

   In contrast to the change in the total number of cases cited over the five decade period, these changes in the mean percentage of string citation are more amenable to this technological explanation. Law clerk allotments, for example, cannot explain the observed changes in string citation. Since the single largest percentage increase in clerk staffing levels arose in 1969, when the permissible number of law clerks was doubled, a “law clerk” influence should significantly affect the 1977 sample. In contrast, however, the 1977 sample showed no statistical difference from either the 1967 or 1957 sample, suggesting that the number of law clerks does not have a determinative influence on the percentage of string citation employed in federal appellate opinions.

   The analysis of the mean number of unique string-cited citations revealed similar results. Between 1957 and 1977, the mean number of unique string-cited opinions increased slightly, from 4.58 to 6.44 citations per case. While this increase over the first twenty years was statistically significant at the 1 percent level, neither change in the included ten-year periods was significant. Overall, however, the trend in these years was slightly upward. The change in mean string citations between 1977 and 1987, however, was remarkably similar to the percentage change detailed above in Table 2: the mean number of citations in that ten-year period diminished substantially, from 6.44 to 4.06 per appellate opinion. Another significant decrease in mean string citation occurred between 1987 and 1997, when the string-cited cases decreased from 4.06 to 2.57 per opinion.[145] 

In sum, these results imply a significant change in the costs of the inputs of opinion production between 1977 and 1997. Specifically, judges appear to produce much less string citation in the 1997 opinions, in both percentage and real numbers. More importantly, as is shown below in Part II.D, judges were not only engaging in fewer string citations, but also analyzing cited cases in significantly greater depth. In effect, computerized research creates a substitution effect in judicial citation consumption: as income increases, judges consume less string citation, and more expository citation. This suggests that string citation is an inferior good—one that judges consume less as income rises.

 

D.   Depth of Citation, 1957–2007

  

   Although this analysis of string citation suggests that string citation practices correlate closely with the use of computer-assisted legal research, whether the observed decrease in string citation is meaningful depends to an extent on the nature of the citation substituted for it. If, for example, judges write single-sentence descriptions of each cited case rather than conglomerate these cases into a string citation, the change in string citation might be significant, but not meaningful. In contrast, if judges are substituting paragraph-long descriptions of cited cases for string citation, it would appear that judicial style has changed in a meaningful fashion.

   In an attempt to measure the import of the substitution effect tangentially observed in the string citation data, the same sample of cases was analyzed to approximate the amount of legal analysis accompanying each citation. Accordingly, each cited case was coded according to the depth of its accompanying analysis, in four increments. Depth I cases represent those cited cases cited only within string citations; Depth II represent those cases cited more directly for a given proposition; and Depth III and IV cases signify those cases cited and discussed “expositorily” within the opinion.[146] In contrast to a change from Depth I to Depth II citation, which may represent only a marginal change in judicial style, a substitution of expository citation for Depth I citation represents a significant transformation in style—from string citation to cases discussed over at least a paragraph.   

   As depicted in Table 3, the increase in expository citation almost directly parallels the decrease in string citation displayed in Tables 1 and 2, with some noteworthy exceptions. Specifically, the greatest change in expository citation occurred between the 1977 and 1987 samples, paralleling the most significant drop in string citation. Numerically, the total percentage of Depth III cases increased 5 percentage points—from 11.5 percent in 1977 to 16.5 percent ten years later. Likewise, Depth IV citations also increased substantially, growing from just over 2 percent of the total citations in 1977 to well over 5 percent of the total citations over the next decade. Taken together, these changes represent an 8.5 percent increase in total expository citation over that period—a change that nearly directly corresponds to the 10.3 percent decrease in unique string-citations from 1977 to 1987. Notably, however, there is no similar parallelism between the 1997 results: while the percentage of string citation declined sizably between 1987 and 1997 (by slightly over 5 percent[147]); the percentage of cases cited expositorily did not change significantly over the same period (a net loss of only 0.1 percent). Yet, expository citation shows a statistically significant change in the decade prior to 1977. Between 1967 and 1977, the percentage of such citation increased a statistically significant 3.3 percent (2.3 percent in Depth III citations, and 1 percent in Depth IV citations).

 

 

   These results suggest that computerized legal research had the greatest effect on judicial citation practices between 1977 and 1987. The substitution of Depth II citation for expository citation between 1967 and 1977 is potentially the result of advances in word processing in that time period, which allowed judges to manageably increase the length of an opinion.[148] Word processing fails to explain the substitution of expository citation for string citation in the 1980s, however. Specifically, the choice to include cases in a string citation is based primarily on the costs of research. Although a judge might be able to credibly appropriate a string cite from the text of another case, investigating whether the cases cited therein are especially analogous requires a substantial amount of additional research.

   The decline in string citation since the 1980s is especially surprising considering the dramatic increase in the federal appellate caseload over the five decades surveyed. Between 1955 and 2005, the number of cases filed in the federal appellate courts increased over 1800 percent: from 3,544 cases in 1955[149] to 68,473 cases in 2005.[150] Although the number of judgeships also increased over the same time span, from 68 in 1955[151] to 179 in 2005,[152] these additional judges did not offset the caseload explosion. Specifically, there were an average of 52.1 cases filed per appellate judge in 1955, and 382.5 per judge in 2005 (an increase of over 630 percent). Despite the overwhelming nature of the caseload increase, however, judges continued to write longer, more expositive opinions between 1957 and 2007.[153]

   In particular, the substitution of string citation for more expositive citation between 1977 and 1987 seems to support the communication theory of judicial citation. Specifically, the pseudo-income effect of cheap, available legal research predicts an increase in consumption of both string and expositive citation, but not necessarily a substitution between the two. One possible explanation for the substitution of expository citation for string citation is that the cost of citing a large number of cases had not only declined in absolute terms, but had declined relative to the cost of citing a lower number of cases. Notably, early computerized research systems did not make opinion writing less arduous (which would arrive only later with the advent of personal computers and sophisticated text-editing programs), but did provide law clerks with a broader array of relevant precedential materials. Thus, judges who had once differentiated their opinions through a parade of precedent could no longer do so, since even a judge with a relatively weak case could string-cite hundreds of cases at least tangentially supporting his decision. Instead, judges began to prefer expository citation, which laid bare a judge’s rationalization and revealed the exact nature of his reliance on “analogous” cases.[154] 

   Although this substitution hypothesis suggests that the reduced costs of legal research have made judicial opinions more honest (since string citation can no longer hide less analogous precedents), it is also possible that the increase in expository citation is merely conspicuous consumption by judges concerned with perceived status. Perhaps expository citation, simply because it is more costly than string citation, creates Veblen effects[155] on judicial consumption of citations. Namely, because expository citation is highly visible and also quite costly, judges might be attracted to such citation not because it necessarily reveals any more substantive information than string citation, but simply because it is costly and overt. Thus conceived, expository citation devolves into an arms race[156] in which judges compete to have the longest and most thoroughly-cited opinions—perhaps because these serve to impress and awe other judges or the legal public through their ostentatious-ness.[157] In this regard, it is important to note, as is demonstrated by Part III, that these stylistic changes are not unique to the aggregate sample of federal appellate opinions, but also appear within the citation patterns of individual judges.

 

 

III. Individual Citation Practices of Two Federal Appellate Judges, 1961–2007

 

   Several commentators have recognized a profound disconnect between the process of judicial decisionmaking and the act of opinion writing. Some suggest that this dichotomy is psychological—arguing that the judicial process can be bifurcated into a “process of discovery” and a “process of justification,” neither of which necessarily informs the other.[158] Others posit a physical disconnect—proposing that although a few appellate judges derive utility from “the intrinsic pleasure of writing” and from “exercising and displaying analytical prowess or other intellectual gifts” through the publication of opinions, most judges now delegate this function almost entirely to their clerks.[159] Both theories suggest that an analysis of the stylistic vagaries of judicial citation patterns is likely to reveal common tendencies in citation, since individual stylistic quirks are unlikely to influence actual citation practices.

   Of course, although the data from comprehensive samples of circuit court opinions arrayed in Tables 1 through 3 above demonstrate the significant effect of declining citation costs on aggregate judicial style, whether this change is indicative of alterations in individual judges’ citation practices is not entirely transparent. Indeed, several institutional accounts of the federal court system may explain the aggregate citation data. Perhaps individual judges, for example, display rather static citation patterns,[160] and the decrease in string citation is a result of a changing judicial workforce. Likewise, perhaps judicial clerks, chosen from among a few elite law schools and generally much younger than the judges they serve,[161] exert determinative influence on overall judicial citation patterns.[162] 

   This Part attempts to answer such objections to the simple economic citation theory through an analysis of two judges over the relevant time period, concluding that although psychological idiosyncrasies might account for a portion of a judge’s citation patterns, the changing cost structure of legal research in the 1970s and 80s had a significant effect on the persistence of such patterns. Accordingly, Part III.A outlines the empirical methodology and reports the citation practices of two long-serving federal appellate judges. Then, Part III.B analyzes these results, hypothesizing that while some individual judges may follow a unique citation style, the vast majority responds directly to changes in the cost of legal research.

 

A.   Data and Methodology

 

   The judge-specific portion of this Article examines the citation patterns of two of the longest-serving federal appellate judges to discover whether the patterns observed in the aggregate sample are also present in the practices of individual judges. Initially, the depth of citation of 291 majority opinions authored by James R. Browning, a Ninth Circuit judge, between 1961 and 2006[163] (a total of 6,233 citations) was evaluated to investigate individual citation patterns. Additionally, 397 majority opinions (4,393 citations) of Judge H. Emory Widener, Jr. of the Fourth Circuit were similarly analyzed.[164] The results obtained appear to generally corroborate the inferences of the main sample, although there are certain distinctive differences.

 

   1.    Judge James R. Browning (Ninth Circuit)

 

   Due to his remarkably long tenure on the federal appellate bench, Judge James R. Browning is a perfect test-case for evaluating the impact of technological research methods on individual citation practices. Judge Browning is one of the longest-serving judges on the federal appellate bench. He began his career on the Ninth Circuit in 1961, when he was appointed by John F. Kennedy after a long and distinguished career in government and private practice.[165] In 1976, he was appointed to the position of Chief Judge of the Ninth Circuit, where he continued to serve until 1988. He continued working as an active judge on the circuit until September 1, 2000, when he took senior status.[166] Thus, Judge Browning served from the shaky inception to the firm establishment of computer-assisted legal research—and these external changes should be reflected in his citation patterns.

   Judge Browning’s specific citation practices, detailed below in Table 4, illustrate that at least some appellate judges altered their citation practices significantly within the last five decades. For example, Judge Browning’s data indicates a steep decline in the amount of string citation over at least the last thirty years. As displayed in Table 4, Judge Browning was once a chronic string-citer: cases cited only in string citations composed more than 37 percent of his early opinions (those between 1961 and 1967). Although the percentage of unique string-cited cases in his opinions declined somewhat in the 1970s (dropping to just below 33 percent in opinions written between 1968 and 1977), the most dramatic changes occurred between 1977 and 1997, when the percentage of string-cited cases in his opinions was approximately cut in half. Specifically, this percentage declined from 32.9 percent in the 1968–1977 opinion sample to 24.6 percent in the 1978–1987 sample; and further to only 15.2 percent in the 1988–1997 sample—significant drops that somewhat parallel the distinctions observed between the 1977, 1987, and 1997 opinions in the aggregate sample.[167] 

 

 

   Judge Browning’s individual citation practices, however, are not necessarily identical to the aggregate sample. His use of string citation continues to decline significantly after 1997, dropping from 15.2 percent in the 1988–1997 sample to only 7.7 percent between 1998 and 2007. While this specific discrepancy might be attributable to the differences in temporal measurement between Judge Browning’s individual samples and the aggregate sample, it is unlikely that this distinction alone will account for all of the variations found between the two analyses. Notably, unlike the aggregate sample, Judge Browning’s use of expository citation does not consistently increase throughout his time on the bench. Instead, his use of Depth III citation climbs from around 4 percent of cases cited in his earliest opinions (1961–1967) to slightly over 16 percent in his more recent opinions (1988–1997) before suddenly declining to 8 percent after 1997. Likewise, his use of Depth IV citation, which was virtually nonexistent in the 1960s, rises only to slightly over 3 percent of his opinions in the 1980s, but remains essentially the same over the next two decades. In contrast, the aggregate sample reports more consistent gains in both types of citation practices over the same period.[168]

   Despite these differences, the overall trends in the citation practices of Judge Browning buttress the implications of the aggregate sample and signify a noteworthy change in the cost structure of opinion writing over the sample period. Indeed, Judge Browning’s string citation practices changed enormously over between his first and last years as an active judge on the federal appellate bench—his unique string citations in the final sample, for example, were only one-fifth of those in the earliest sample. Barring a significant change in judicial ideology, such differences indicate larger economic factors were influencing the judge’s citation practices over his many years of service. Perhaps it is not coincidental that Judge Browning was among the most stalwart advocates of advancing computer technology in the Ninth Circuit—since his opinions appear to validate evidence of a changing production cost structure.[169]

 

 

Figure 2

Percentage of String-Cited Cases,

by Year and by Individual Opinions of Judge Browning

 

   Figure 2’s graphical depiction of Judge Browning’s string citation patterns portrays the exact nature of these remarkable variations. Although the changes are somewhat subtle, the graph does illustrate the gradual waning of string citation in the judge’s opinions. Specifically, although unique string citation in Judge Browning’s 1960s opinions seem to cluster between 30 and 60 percent, after the early 1980s only one opinion contained over 50 percent string citation. In fact, string citation declines continuously throughout the 1980s and 1990s—so that by the mid-1990s Judge Browning’s opinions show string citation clustering around the 10 to 15 percent levels. Remarkably, although many opinions throughout the sample contained no string citation whatsoever, those opinions with enormous levels of string citation are almost nonexistent after the late-1970s.

 

   2.    Judge H. Emory Widener, Jr. (Fourth Circuit)

 

   Like Judge Browning, Judge H. Emory Widener was one of the longest serving appellate judges in history. He was appointed to the federal appellate bench in 1972, by Richard Nixon, after three years on the federal district court for the Western District of Virginia.[170] He served for thirty-five years on the Fourth Circuit and was an active member of the bench until 2007, when he took senior status, and was at the time of his death in 2007 the longest-serving active member of that court. Over that time, he published a large number of opinions—which provide a wealth of information about his citation practices.

   Judge Widener’s citation patterns, found in Table 5 below, almost directly parallel those of the aggregate sample. Beginning in 1972, his use of string citation steadily declines over the next three decades: dropping from over 21 percent in the 1972–1977 sample to slightly under 9 percent in the 1998–2007 sample. Unlike Judge Browning, and similar to the results of the aggregate sample, Judge Widener’s use of expository citation (especially Depth IV citation) progressively increases throughout his time on the bench. His use of Depth III citation increases from 9.4 percent in the 1972–1977 sample to over 15 percent in the 1998–2007 sample (almost exactly corresponding to the results in Part I, which found increases from around 9 to 16.1 percent over the same period).[171] Likewise, the judge uses Depth IV citation more frequently in later years: Depth IV citation swells from less than 1 percent of cited cases to almost 5 percent of cited cases between 1972 and 2007.

 

 

   Interestingly, there are also some idiosyncrasies within Judge Widener’s citation patterns that are not found in the aggregate sample. For example, Judge Widener’s use of Depth II citation does not change significantly over his years on the bench—ranging between 68 and 71 percent in all four samples. Also, his use of expository citation peaks between 1988 and 1997, and declines marginally in between 1998 and 2007 (although this may simply be the result of age effects on citation). Also, as graphically portrayed below in Figure 3, Judge Widener’s rate of string citation declines rather suddenly after 1980—quite unlike the slow decline in string citation found in Judge Browning’s opinions. Specifically, there are very few differences between Judge Widener’s mid-1980s and mid-2000s opinions—all seem to cluster between the 10 and 20 percent levels. Most of the differences between the 1978–1987 sample and subsequent samples, in fact, appear to arise from the amount of opinions without any string citation, which seem to increase substantially in the 1990s and 2000s.[172]

 

 

Figure 3

Percentage of String-Cited Cases,

by Year and by Individual Opinions of Judge Widener

 

B.       The Death of Individual Style?

 

   This large amount of data on individual judge’s citation practices over the last fifty years illustrates that the cost of citation has a measurable effect on citation patterns and indeed, on judicial literary style.[173] As Judge Posner has suggested, perhaps this is the result of the influence judicial clerks exert on the opinion-writing process. Posner argues that “although . . . delegation of opinion drafting to law clerks may result in a change of literary style with every change of law clerks, the dominant effect is stylistic uniformity rather than variety.”[174] While Posner attributes this to the “uniform educational experience” of law schools and the concomitant indoctrination into the ponderous writing style of law reviews,[175] the data on citation of individual judges suggests that this “culture” is not temporally uniform, and is based to a large extent on the costs of citation. Law review articles, similarly to judicial opinions, have swelled in both subject matter and citation over the years as law professors gained increased access to other legal scholarship and technological advancements made the editing process less onerous.[176] 

   As the data demonstrates, the effect of computer-assisted legal research on citation patterns may actually be conforming. For example, consider the differences between the citation patterns of Judge Browning and Judge Widener before and after the advent of computerized legal research systems. Before 1977, Judge Browning’s Depth I and Depth II citation patterns are statistically distinct from Judge Widener’s—but afterwards both judges appear to move towards a common mean. In fact, the two judges’ citation patterns in the 1978–1987 sample show only one significant difference (in Depth I citation), and their citation patterns in the 1988–1997 sample show none.[177] In this sense, each judge has become more similar in citation over their years on the bench. Whether this effect is beneficial or worrisome depends on its underlying causes: perhaps, for example, appellate judges simply adapt to the opinion form most accessible to their audience; or perhaps, as Judge Posner suggests, computerized legal research has an indoctrinating effect on law clerk (and hence judicial) writing style. As Part IV proves, however, judicial citation practices still contain some useful information, despite this tendency towards uniformity.      

 

IV. Dissenting Citation Practices in the Federal Appellate Courts, 1957–2007

 

   If citations are a form of communication, a microeconomic approach would indicate that as the cost of legal research declines, the effects on citation amount and style will not necessarily be uniform across cases. Rather, the effect will be most pronounced in high-stakes cases, where the value of the communication is greatest. If, as some commentators have proposed, appellate judges are more likely to use a dissenting opinion to communicate where they might have a censorial “whistleblowing” effect on the majority’s decision,[178] using citation as communication might also be particularly manifest in those dissenting opinions in which a judge disagrees with the majority on an ideological basis. In quantitative terms, citation signaling might be especially likely to occur whenever the dissenter is of the opposite ideological affiliation than the majority opinion-writer. Alternatively, however, dissenting judges might communicate their strong disagreement with the majority’s reasoning not through the style of their citation, but through the citations’ content. Under this theory of citation signaling, as the cost of finding arguably applicable precedent decreases, dissenting judges may be especially likely to cite more cases and more diverse cases in ideological dissents.

   In order to explore the differences in citation patterns between ideological and non-ideological dissents, this Part empirically analyzes a set of 600 dissenting opinions. Part IV.A sets out the empirical methodology, Part IV.B analyzes dissenting citation style, and Part IV.C evaluates the amount of dissenting citation—revealing some statistically significant differences between ideological and non-ideological citation patterns.

 

A.   Data and Methodology

 

   Employing the same set of federal appellate opinions used in Part II, this Article used the time-honored method of coding political ideologies to estimate the effects of ideological valence on the citation practices of dissenting judges. First, all judges in the set of 600 dissenting opinions, collected at ten-year intervals from 1957 to 2007, were coded either “D” or “R” depending on the party of the judge’s appointing president. Then, the dataset was examined by counting the number of cases, as found in Westlaw’s Table of Authorities, which appeared uniquely[179] in the dissenting opinion in each of the relevant cases. Each set of 100 dissenting opinions was divided based on the relationship between the ideological coding of the majority and dissenting judges—specifically, whether the political affiliation of the majority judge was “Same” or “Opposing” relative to the dissenting judge’s political affiliation.

 

 

 

B.   Citation Style

 

   The interesting results of the analysis of dissenting citation style indicate that signaling to the Supreme Court or a future en banc panel is unlikely to explain the miniscule differences in citation style between ideological and non-ideological dissents. There were virtually no statistically significant differences in the level of string citation or expository citation between ideological and non-ideological dissenting opinions. The only statistically significant difference, between Depth IV citation in 1967 ideological and non-ideological dissenting opinions, was almost microscopic: specifically, Depth IV citation in that sample rose from 0.7 percent to only 2 percent when the dissenting judge is of the opposite political ideology than the majority judge.

   More importantly, there were no significant trends in citation patterns observed over time. Unlike the signaling theory predicts, the difference between string citation levels in ideological and non-ideological opinions does not decrease over the five decades covered by the samples: instead, it fluctuates rather randomly over the period. Likewise, expository citation does not become especially prominent in ideological dissenting opinions as computer-assisted research proliferates.

   

C.       Citation Amount

 

   Although no significant differences emerged between the citation styles of judges dissenting from their ideological opposites, several interesting differences between the amounts of citations in dissenting opinions surfaced upon further analysis. Specifically, there appear to be some interesting differences both between the actual amount of cited cases in dissenting opinions and the percentage of such citations relative to the cases cited in the opinion as a whole—differences that appear linked with the ideological correlation between the majority and dissenting judge.

   The mean number of unique cases cited in dissent, outlined in Table 6, demonstrates some of these significant differences between the structure of ideological and non-ideological dissents. Notably, the difference between those dissenting opinions in which the majority and dissenting judges are of the same political party, and those opinions in which the majority and dissenting judges are of opposing political parties, is statistically significant in three of the six sample years: 1957, 1997, and 2007. In 1957, while non-ideological dissents averaged slightly under two unique cases cited in dissent, ideological dissents averaged over double that number (slightly more than five). Similar differences were found in the 1997 and 2007 samples, where ideological dissents cited an average of three more unique cases than non-ideological dissents.

 

 

   In the remaining three samples, however, no such distinction between dissenting citations was observed. In 1967, although ideological dissents averaged around 1.5 unique cited cases more than non-ideological dissents, this number was not statistically significant due to the large variations within the sample. Similar results were observed in the 1987 sample, where a 1.15 average difference in cited cases between dissent types was not statistically significant. In fact, in the 1977 sample, the mean number of cases cited in non-ideological dissents was slightly higher than the mean in ideological dissents (albeit not by a statistically significant amount). The difference observed in the 1957, 1997, and 2007 samples, therefore, is not temporally uniform.

   Although these average numbers do not necessarily denote considerable underlying trends, controlling for the length of the analyzed opinions also reveals significant differences between ideological and non-ideological dissents. Table 7, below, depicts the mean number of unique cases as a percentage of the total number of cases cited in the majority and dissenting opinion. As in Table 6, the 1957, 1997, and 2007 datasets reveal large and significant differences in unique dissenting citations relative to the party of the majority judge. In 1957, the percentage of unique cases cited in cases where the majority judge is appointed by a president of the opposite political party is over 10 percentage points higher than when the majority writer is of the same political party. Likewise, the difference between the two types of opinions is 8.7 percentage points in 1997 and 10.5 percentage points in 2007—results which are both statistically significant to at least the 5 percent level. Also significant are the total results from all 600 analyzed opinions, which reveal that over the past fifty years, the percentage of unique citations in ideological dissents is on average 6 percentage points higher than in non-ideological dissents.

 

 

   The pattern of these results is also rather significant (as shown below in Figure 4). Specifically, the differences between ideological and non-ideological dissenting citation follow an almost parabolic curve, slowly waning from an 85 percent increase in citation in ideological opinions in 1957 to a 16 percent decrease in 1977, before gradually inflating to a nearly 69 percent increase in 2007.

   Unfortunately, the parabolic pattern revealed in Table 7 does not validate the communication hypothesis of dissent citation. As the cost of legal research decreases, and dissenting judges are more likely to be able to find arguably analogous precedent, the difference in citation amount between ideological and non-ideological opinions should increase accordingly. Likewise, these results do not correspond with the increases in total citation found in Table 1. Table 1 reveals that mean total citation in opinions containing a corollary dissenting opinion increased dramatically in 1977 (from 18.8 to 25.2 citations per opinion), which may explain the reduction in signaling value of dissent citation. Increases in total citation, however, fail to rationalize the 2007 results. Between 1997 and 2007, mean total citation also increased substantially (from 24.5 to 33.2 citations per opinion), and yet, unlike in 1977, there is a significant difference between ideological and non-ideological dissenting citations.[180]

 

 

Figure 4

Ideological and Non-Ideological Dissents,

by Year and Percentage of Cases Uniquely Cited in Dissent

 

   One alternative explanation is that judicial partisanship is a temporal phenomenon—and that such partisanship has been steadily increasing since the 1970s. Essentially, the results in Table 7 measure the amount of disagreement between the majority and the dissenting judges over the applicable precedent. A greater number of unique cases cited in dissent indicate that the dissenting judge disagrees not only with the majority’s characterization of its cited cases, but exactly what cases are even applicable to the relevant factual circumstances. Thus, perhaps the distinctions between ideological and non-ideological dissents correspond to those periods in American legal history in which substantial conflicts arose between judges of differing political backgrounds over the direction of American law. For example, the large distinction in 1957 (and to a lesser extent, 1967) could represent the legal struggle of the civil rights movement illustrated in Brown v. Board of Education.[181] Likewise, the growing partisanship in the 1990s and 2000s may represent the increasing political divide over issues such as the right to privacy and federalism.

   In sum, this sort of analysis provides a fertile area for future research. Given that such differences exist in citation patterns between ideological and non-ideological dissents, perhaps future studies could measure these differences in the context of specific subject matters (such as constitutional law) to discover how new legal doctrines are slowly integrated into judicial reasoning. Also, such future studies could look more closely at ideological differences between majority and dissenting judges[182] to explore whether those judges on opposite ends of the political spectrum are more likely to disagree on the applicable precedent than less ideologically-distinct judges. Importantly, the results in this Article show that such studies are likely to be successful, since the political affiliation of judges appears to affect not just the ultimate holding of an opinion, but that opinion’s citation content (and amount).

 

V. Conclusion

 

   This Article proposes that a simple microeconomic approach can describe judicial citation practices over the last fifty years. It argues that judges (or perhaps their clerks) use citations primarily as a communication device, and that the cost of legal research is intimately connected with the effectiveness of this communication (and therefore with judicial citation patterns). The empirical results in this Article not only demonstrate the effectiveness of the microeconomic approach in describing judicial opinion style, but also provide a foundation for future research into the effects of judicial ideology on citation practices.