Algorithmic fairness in predictive policing
The growing use of algorithms to predict risk in policing has raised concerns that such systems may amplify existing societal biases. This research adopts a two-phase approach, combining a systematic review with an empirical audit and mitigation of age-related bias in a deployed system. It examines the Chicago Police Department's Strategic Subject List (SSL), which scores an individual's probability of involvement in a shooting incident, either as an offender or a victim. The review finds that existing work concentrates on racial bias, while the other three attributes, age, gender, and socio-economic status, are studied far less. The authors selected age for their empirical audit, which found all 118,146 individuals under 30 in the dataset classified as high risk. They introduce Conditional Score Recalibration (CSR), which, applied alongside the established Class Balancing method, significantly reduces age bias without any loss of model accuracy.
Research Objectives
- To review which fairness metrics, forms of bias, and mitigation strategies existing research on policing and recidivism algorithms has used.
- To identify the gap left by research that overlooks protected attributes other than race, particularly age, gender, and socio-economic status.
- To audit and mitigate age bias in a deployed predictive policing system, using the Chicago Police Department Strategic Subject List dataset.
- To introduce Conditional Score Recalibration and assess whether improving fairness must come at the cost of accuracy.
Methodology
- A systematic review following PRISMA, covering work published from 2015 onward. From 587 initial results, 15 studies were analysed.
- The Chicago Police Department SSL dataset, covering 398,684 individuals between August 2012 and July 2016. Scores range from 0 to 500, with scores above 250 treated as high risk. After preprocessing, 170,694 records and 12 features remained.
- Random Forest, Logistic Regression, and Gradient Boosting were trained, with Random Forest selected. Performance was measured with Accuracy and F1.
- Fairness was measured with three metrics, Demographic Parity, Equality of Opportunity, and Average Odds Difference, each comparing outcomes between two groups, with values between -0.1 and 0.1 treated as acceptable.
- CSR reassigns individuals scoring 250 to 350 as low risk if they have no prior arrest for violent offenses, no prior arrest for narcotic offenses, and no history of being a shooting victim. Class balancing was then applied before training.
Key Findings
- Of the 15 studies reviewed, 12 examined racial bias, while 4 examined gender, 3 examined age, and 2 examined socio-economic status. Almost all relied on United States data.
- Police operational behaviour is a major driver of discrimination. Deploying more officers to an area produces more arrests, which raises reported crime figures, which in turn justifies further deployment. This cycle arises even where no algorithm is involved.
- The SSL dataset carries severe age bias. All 118,146 individuals under 30 were classified as high risk, and among the 127,513 individuals never arrested or shot, around 90,000 were still deemed high risk.
- Before mitigation the model passed all three fairness thresholds for race but clearly failed on age (Demographic Parity 0.8517, Equality of Opportunity 0.7616, Average Odds Difference 0.3349). After CSR with class balancing these fell to 0.3128, 0.1521, and 0.02024, while accuracy rose from 0.8314 to 0.9014.
- Mitigating age bias affected race. Demographic Parity for race rose from 0.09923 to 0.1170, moving slightly outside the acceptable range.
Recommendations
- Embed human review in algorithmic decision-making, using structured group assessment, since reviewers carry their own biases.
- Diversify data sources beyond the United States, as legal systems and population structures differ between countries.
- Widen bias auditing to cover age, gender, and socio-economic status, the attributes existing literature has largely neglected.
- Design mitigation to suit the context of each system rather than applying a single off-the-shelf approach.
- Future research should segment age more finely than a single cut at 30, test the method on data from other regions, and develop CSR further, since its preset conditions may not capture every dimension of individual risk.
Key Takeaways
- A deployed risk scoring system can carry severe bias that goes unnoticed until it is systematically audited.
- Fairness and accuracy need not be traded off. Age bias fell substantially while accuracy rose from 0.8314 to 0.9014.
- Mitigating bias on one attribute can disturb another, so audits should examine all attributes together. For Thailand, any agency adopting individual risk scoring should first establish which variables drive the score and whether any group is being classified wholesale.
References
Almasoud, A. S., & Idowu, J. A. (2025). Algorithmic fairness in predictive policing. AI and Ethics, 5, 2323–2337. https://doi.org/10.1007/s43681-024-00541-3
Full text (Open Access): https://link.springer.com/article/10.1007/s43681-024-00541-3
