Algorithms, Bias, and the Female Defendant

When the state outsources judgment to algorithms, it does not eliminate human bias — it automates it. Predictive policing tools, recidivism risk scores, and pretrial detention algorithms are being deployed across criminal justice systems worldwide, including in India's burgeoning digital governance infrastructure. These systems promise objectivity, efficiency, and fairness. What they deliver is a mirror of the data they were trained on — data generated by a justice system that has always been stacked against women, people of color, and the poor.

"The algorithm doesn't discriminate. It just faithfully reproduces the discrimination already embedded in the data it was given."

The Promise and the Problem

Risk assessment algorithms like COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) in the United States, or similar tools being piloted in Indian courts, analyze factors like criminal history, employment status, age, and neighborhood to predict whether a defendant is likely to reoffend or fail to appear in court. Judges use these scores to make decisions about bail, sentencing, and parole.

The appeal is obvious. Human judges are subject to fatigue, prejudice, and inconsistency. An algorithm applies the same criteria to every case. But this framing misunderstands the nature of bias. Bias does not only enter the system at the point of decision — it enters at the point of data collection. If Black defendants are arrested at higher rates due to over-policing, the algorithm learns that race correlates with criminality. If women who report domestic violence are more likely to have police records for "disturbance," the algorithm learns that women who seek help are dangerous.

**ProPublica's** 2016 investigation of COMPAS found that the tool was nearly twice as likely to falsely flag Black defendants as future criminals compared to white defendants. The gender dimension of algorithmic bias has received less attention but is no less significant. Research by **Safiya Noble** and others has documented how AI systems consistently reproduce gendered assumptions — treating women's behavior as more deviant, more emotional, and more deserving of punishment than identical male behavior.

Gendered Data, Gendered Outcomes

Criminal justice data is not gender-neutral. It reflects a system that has historically treated women's victimization as less serious and women's offending as more aberrant. When algorithms learn from this data, they learn these same hierarchies.

Consider domestic violence cases. Women who kill abusive partners in self-defense often have longer criminal records than their abusers — because they fought back, called police, or were arrested in mutual-assault situations where the actual perpetrator was not identified. A risk assessment algorithm that weights criminal history will score these women as higher risk, regardless of the context that produced their records.

In India, where **Section 498A** of the Indian Penal Code (cruelty by husband or relatives) is frequently invoked — and frequently misused, according to men's rights groups — algorithmic tools trained on FIR data could easily develop anti-women bias. Women who file cruelty complaints are already stigmatized as "vindictive"; an algorithm that flags them as "repeat litigants" would formalize this stigma into a quantifiable risk score.

Predictive Policing and the Gendered Geography of Crime

Predictive policing tools like PredPol (now Geolitica) analyze historical crime data to predict where crimes are likely to occur, directing police resources to specific neighborhoods. The problem is circular: areas that have been heavily policed generate more crime data, which leads to more policing, which generates more data. The algorithm never learns that a "high-crime area" might simply be a "high-policing area."

For women, this creates specific dangers. Domestic violence and sexual assault are systematically underreported and underpoliced. Algorithms trained on reported crime data will undercount these offenses while overcounting street crime in poor neighborhoods. The result is a system that allocates resources away from the crimes that disproportionately affect women — and toward the crimes that disproportionately affect men of color.

In Indian cities, where **eve-teasing**, stalking, and domestic violence are endemic but rarely result in formal police complaints, predictive policing tools would be working with catastrophically incomplete data. The algorithm would not predict where women are unsafe — it would predict where the police have historically paid attention, which is a very different thing.

The Accountability Gap

When a human judge makes a biased decision, there is at least the possibility of appeal, review, and accountability. When an algorithm makes a biased decision, the accountability structure collapses. The developers claim trade secret protection for their proprietary models. The courts claim they are merely using the algorithm as one factor among many. The defendants lack the technical expertise to challenge the methodology.

This **accountability vacuum** is particularly dangerous for marginalized communities. A woman challenging a risk score that classifies her as high-risk would need to understand the algorithm's methodology, access its training data, and demonstrate statistical bias — all without the resources that the state and private companies can bring to bear.

In India, where judicial infrastructure is already strained and digital literacy varies enormously, the deployment of algorithmic tools in criminal justice is likely to outpace the development of oversight mechanisms. The National Crime Records Bureau and state police forces are already using data analytics in ways that lack transparency or public scrutiny.

Resisting the Black Box

The feminist response to algorithmic bias in criminal justice is not to reject technology outright — it is to demand transparency, accountability, and a seat at the table. Algorithms used in criminal justice should be subject to mandatory bias audits, conducted by independent researchers with access to the full training data and methodology. Affected communities should have standing to challenge algorithmic decisions. And the underlying data — the arrests, the convictions, the sentencing patterns — must be confronted honestly, rather than laundered through the appearance of objectivity.

"An algorithm is not a solution to bias. It is a technology that requires the same scrutiny we would apply to any other instrument of state power."

The danger is not that computers will develop patriarchal consciousness. The danger is that we will hand them a patriarchal world and call the result progress.