Jurimetrics

List of Applications

• Accounting fraud detection (Benford's law)
• Airline deregulation
• Analysis of police stops (Negative binomial regression)
• Ban the Box legislation and subsequent impact on job applications
  • Statistical discrimination (economics)
• Calorie labeling mandates and food consumption
  • Risk compensation
• Challenging election results (Hypergeometric distribution)
• Condorcet's jury theorem
• Cost-benefit analysis of renewable portfolio standards for greenhouse gas abatement
• Effect of compulsory schooling on future earnings
• Effect of corporate board size on firm performance
• Effect of damage caps on medical malpractice claims
• Effect of a fiduciary standard on financial advice
• False conviction rate of inmates sentenced to death
• Legal evidence (Bayesian network)
• Impact of "pattern-or-practice" investigations on crime
• Legal informatics
• Ogden tables
• Optimal stopping of clinical trials
• Peremptory challenges in jury selection
• Personality predictors of antisocial behavior
• Predictive policing
• Predictors of criminal recidivism
• Prevalence of Caesarean delivery and malpractice claims risk
• Prosecutor's fallacy (People v. Collins)
• Reference class problem
• Judicial efficiency

Applications

Gender quotas on corporate boards

In 2018, California's legislature passed Senate Bill 826, which requires all publicly held corporations based in the state to have a minimum number of women on their board of directors. Boards with five or fewer members must have at least two women, while boards with six or more members must have at least three women.

Using the binomial distribution, we may compute what the probability is of violating the rule laid out in Senate Bill 826 by the number of board members. The probability mass function for the binomial distribution is:$${\displaystyle \mathbb{P}(X=k)=(\genfrac{}{}{0ex}{}{n}{k})p^k(1-p{)}^{n-k}}$$where ${\displaystyle p}$ is the probability of getting ${\displaystyle k}$ successes in ${\displaystyle n}$ trials, and $(\genfrac{}{}{0ex}{}{n}{k})$ is the binomial coefficient. For this computation, ${\displaystyle p}$ is the probability that a person qualified for board service is female, ${\displaystyle k}$ is the number of female board members, and ${\displaystyle n}$ is the number of board seats. We will assume that ${\displaystyle p=0.5}$.

Depending on the number of board members, we are trying compute the cumulative distribution function:$${\displaystyle \{\begin{array}{ll}\mathbb{P}(X\le 1)=(1-p{)}^n+np(1-p{)}^{n-1},& n\le 5\\ \mathbb{P}(X\le 2)=\mathbb{P}(X\le 1)+\frac{n(n-1)}{2}{p}^2(1-p{)}^{n-2},& n>5\end{array}}$$With these formulas, we are able to compute the probability of violating Senate Bill 826 by chance:

As Ilya Somin points out, a significant percentage of firms - without any history of sex discrimination - could be in violation of the law.

In more male-dominated industries, such as technology, there could be an even greater imbalance. Suppose that instead of parity in general, the probability that a person who is qualified for board service is female is 40%; this is likely to be a high estimate, given the predominance of males in the technology industry. Then the probability of violating Senate Bill 826 by chance may be recomputed as:

Screening of drug users, mass shooters, and terrorists

In recent years, there has been a growing interest in the use of screening tests to identify drug users on welfare, potential mass shooters, and terrorists. The efficacy of screening tests can be analyzed using Bayes' theorem.

Suppose that there is some binary screening procedure for an action ${\displaystyle V}$ that identifies a person as testing positive ${\displaystyle +}$ or negative ${\displaystyle -}$ for the action. Bayes' theorem tells us that the conditional probability of taking action ${\displaystyle V}$, given a positive test result, is:$${\displaystyle \mathbb{P}(V|+)=\frac{\mathbb{P}(+|V)\mathbb{P}(V)}{\mathbb{P}(+|V)\mathbb{P}(V)+\mathbb{P}(+|\sim V)\left[1-\mathbb{P}(V)\right]}}$$For any screening test, we must be cognizant of its sensitivity and specificity. The screening test has sensitivity ${\displaystyle \mathbb{P}(+|V)}$ and specificity $\mathbb{P}(-|\sim V)=1-\mathbb{P}(+|\sim V)$. The sensitivity and specificity can be analyzed using concepts from the standard theory of statistical hypothesis testing:

• Sensitivity is equal to the statistical power ${\displaystyle 1-\beta }$, where ${\displaystyle \beta }$ is the type II error rate
• Specificity is equal to ${\displaystyle 1-\alpha }$, where ${\displaystyle \alpha }$ is the type I error rate

Therefore, the form of Bayes' theorem that is pertinent to us is:$${\displaystyle \mathbb{P}(V|+)=\frac{(1-\beta )\mathbb{P}(V)}{(1-\beta )\mathbb{P}(V)+\alpha \left[1-\mathbb{P}(V)\right]}}$$Suppose that we have developed a test with sensitivity and specificity of 99%, which is likely to be higher than most real-world tests. We can examine several scenarios to see how well this hypothetical test works:

• We screen welfare recipients for cocaine use. The base rate in the population is approximately 1.5%, assuming no differences in use between welfare recipients and the general population.
• We screen men for the possibility of committing mass shootings or terrorist attacks. The base rate is assumed to be 0.01%.

With these base rates and the hypothetical values of sensitivity and specificity, we may calculate the posterior probability that a positive result indicates the individual will actually engage in each of the actions:

Even with very high sensitivity and specificity, the screening tests only return posterior probabilities of 60.1% and 0.98% respectively for each action. Under more realistic circumstances, it is likely that screening would prove even less useful than under these hypothetical conditions. The problem with any screening procedure for rare events is that it is very likely to be too imprecise, which will identify too many people of being at risk of engaging in some undesirable action.

Historical applications

Jurimetrics utilizes many statistical methods to analyze judicial behavior, and this occurs through uncovering patterns in decision-making and using them to identify potential biases in judgements that are passed. For instance, statistical analysis can forecast the outcomes of cases, providing insights into expected resolutions based on historical data. Jurimetrics is also used to evaluate litigation trends, optimize legal strategies, and improve the efficiency of legal proceedings.

One example of an application of jurimetrics is through resource allocation within court systems, where data analytics are used to identify potential difficulties and suggests improvements. Another example is the analysis of disparities within sentencing. This allows policymakers to address the inequities within legal practices. These emphasize the role of jurimetrics in the legal system, as a way to bridge quantitative analysis, and equitable judicial processes.

List of methods

• Bayesian inference
• Causal inference
  • Instrumental variables
• Design of experiments
  • Vital for epidemiological studies
• Generalized linear models
  • Ordinary least squares, logistic regression, Poisson regression
• Meta-analysis
• Probability distributions
  • Binomial distribution, hypergeometric distribution, normal distribution
• Survival analysis
  • Kaplan-Meier estimator, proportional hazards model, Weibull distribution

Bayesian analysis of evidence

Bayes' theorem states that, for events ${\displaystyle A}$ and ${\displaystyle B}$, the conditional probability of ${\displaystyle A}$ occurring, given that ${\displaystyle B}$ has occurred, is:$${\displaystyle \mathbb{P}(A|B)=\frac{\mathbb{P}(B|A)\mathbb{P}(A)}{\mathbb{P}(B)}}$$Using the law of total probability, we may expand the denominator as:$${\displaystyle \mathbb{P}(B)=\mathbb{P}(B|A)\mathbb{P}(A)+\mathbb{P}(B|\sim A)[1-\mathbb{P}(A)]}$$Then Bayes' theorem may be rewritten as:$${\displaystyle \begin{array}{cc}\mathbb{P}(A|B)& =\frac{\mathbb{P}(B|A)\mathbb{P}(A)}{\mathbb{P}(B|A)\mathbb{P}(A)+\mathbb{P}(B|\sim A)[1-\mathbb{P}(A)]}\\ & =\frac{1}{1+\frac{1-\mathbb{P}(A)}{\mathbb{P}(A)}\frac{\mathbb{P}(B|\sim A)}{\mathbb{P}(B|A)}}\end{array}}$$This may be simplified further by defining the prior odds of event ${\displaystyle A}$ occurring ${\displaystyle \eta }$ and the likelihood ratio ${\displaystyle \mathcal{L}}$ as:$${\displaystyle \eta =\frac{\mathbb{P}(A)}{1-\mathbb{P}(A)},\mathcal{L}=\frac{\mathbb{P}(B|A)}{\mathbb{P}(B|\sim A)}}$$Then the compact form of Bayes' theorem is:$${\displaystyle \mathbb{P}(A|B)=\frac{1}{1+(\eta \mathcal{L}{)}^{-1}}}$$Different values of the posterior probability, based on the prior odds and likelihood ratio, are computed in the following table:

If we take ${\displaystyle A}$ to be some criminal behavior and ${\displaystyle B}$ a criminal complaint or accusation, Bayes' theorem allows us to determine the conditional probability of a crime being committed. More sophisticated analyses of evidence can be undertaken with the use of Bayesian networks.

Emerging Trends in Jurimetrics

As many other fields, the changes to jurimetrics have been dynamic due to technological advancements. The integration of artificial intelligence(AI) into legal processes has been an emerging trend. Machine learning algorithms, an AI powered tool, have been used frequently to analyze legal texts, predict case outcomes, and provide data-focused insights to legal employees.

Technological advancements such as AI have been used in creating legal analytics platforms. They can review large amounts of case law, and identify patterns that assist in crafting legal arguments. These innovations improve decision-making processes by reducing the likelihood of human error, but also increase the efficiency of legal research.

For example, recent studies highlight the efficiency of ML in analyzing complex datasets, such as those found in healthcare or legal domains, with high accuracy. One application discussed by Christian Garbin, Nicholas Marques, and Oge Marques (2023) involves the use of ML models to identify specific patterns in datasets characterized by class imbalances. The article discusses datasets related to opioid use disorder, and how judgements passed in legal environments have been dependent on these datasets that are connected closely to class imbalances.

Despite many advancements, the integration of AI into jurimetrics presents challenges. Garbin, Marques, and Marques emphasize that many studies that use machine learning algorithms fail to transparently document essential steps, such as data preprocessing, hyperparameter tuning, or the criteria used for splitting training and test sets.

Garbin, Marques, and Marques recommend prioritizing interpretable models unless the performance gap justifies the use of less transparent algorithms. Since legal decisions have high-stakes, interpretable models(logistic regression or decision trees) are often preferred over more complex "black-box" models. Often, these "black-box" models have higher predictive accuracy, but the interpretability is a central and ethical concern.

History of Jurimetrics

The term "jurimetrics" was created in 1949 by Lee Loevinger. It was defined as the use of quantitative methods to the study of law. Initially, jurimetrics was specifically focused on the theoretical exploration of statistical techniques on legal systems.

Over time, the field evolved. In the mid-20th century, jurimetrics began to gain traction as researchers continued to explore the field and its potential for improving legal analysis. Early foundational studies created a roadmap for actually integrating the practice into the legal field. By the late 20th century, jurimetrics expanded to include applications such as evaluating the reliability of forensic evidence and modeling litigation outcomes.

In today's world, jurimetrics is recognized as a tool for the modern day legal system. It bridges the gaps between economics, data science, and the law.

Ethics of Jurimetrics

In 2021, Abigail Z. Jacobs and Hanna Wallach released a study regarding "computational systems, and how they often involve unobservable theoretical constructs, such as socioeconomic status, teacher effectiveness, and risk of recidivism". "Computational systems have long been touted as having the potential to counter societal biases and structural inequalities, yet recent work has demonstrated that they often end up encoding and exacerbating them instead".

An example of the ethical concerns in jurimetrics comes from risk assessment models used in the U.S. justice system, particularly seen in the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) tool. COMPAS is developed by Northpointe(now Equivant), and was built to evaluate a defendant's likelihood of recidivism through the analysis of various factors derived from official records and interviews. The factors are grouped into four dimensions: prior criminal history, associations with criminals, drug involvement, and indicators of juvenile delinquency.

The risk assessment model then uses the factors in a regression model to generate a recidivism risk score, scaled from one to ten, with ten indicating the highest risk. According to the model, recidivism is defined as a new misdemeanor or felony arrest within two years. However, the specific mathematical methodology that COMPAS uses remains private, which has raised concerns regarding transparency. Subsequent investigations, such as those by Angwin et al., have critiqued the model for potential biases and their ethical implications.