An algorithm is a set of steps that leads to a conclusive result. Both humans and computers can process algorithms – even simple arithmetic is an algorithm. Computers use complex algorithmic formulas that reference large quantities of data in order to make automated decisions.
Governments, employers, insurance companies, and health care providers are dramatically increasing their reliance on automatic decision-making software. Decisions as diverse as evaluating risk during criminal sentencing, deciding where to prioritize police resources, sorting resumes, ranking patients by greatest medical need, and determining eligibility and quantity of public benefits may all rely on computer automation.
Neutral or Biased?
As automated decision-making increases, so does litigation alleging harm to people, due to flaws or biases in the underlying algorithms.
There is an ideological split between people who see computer-automated decision-making as the neutral arbiter of the future, and those who view computer-automated decision-making as a way for governments and corporations to abdicate responsibility for bias, because a human did not make the decision.
At one extreme, computers lack the mens rea necessary for bias, and are therefore more trustworthy than humans. At the other extreme, computing technology has become a sort of “Frankenstein’s monster,” one that is destructive to human agency and too vast to control.
K.W. ex rel. D.W. Armstrong
Regardless of varying opinions on the harms of technology, the lawsuits are real, and both plaintiff and defense attorneys need to understand possible claims.
One introductory resource is Poverty Lawgorithms: A Poverty Lawyer's Guide to Automated Decision-Making Harms on Low-Income Communities by Michele Gilman. This guide maintains that lawyers do not need to master software programming in order to effectively litigate claims of algorithm bias. Gilman compares a requisite technical knowledge of algorithms to that of a tort lawyer’s knowledge of pharmaceutical side effects. Most lawyers are neither doctors nor programmers, yet are still able to understand how a good or service might cause harm and what law might apply as a remedy.
The “Poverty Lawgorithms” guide references K.W. ex rel. D.W. v. Armstrong,1 a federal class action suit involving Idaho Medicaid benefits for developmentally disabled individuals. The Idaho Department of Health and Welfare (IDHW) administers Medicaid for disabled individuals to purchase supports for living in the community rather than an institution. The Idaho program is similar to the IRIS Medicaid program administered by the Wisconsin Department of Health Services. In both programs, an individual is assessed for eligibility and need, and then given an individualized budget with which they choose and purchase the necessary services.
IDHW used a budget tool (algorithmic software) that reduced budgets for the plaintiffs. The court found the notice of reduction in funding, as well as the budget tool itself, to be lacking. IDHW and the plaintiffs eventually reached a settlement whereby IDHW would replace the budget methodology and offending software, evaluate the new budget tool on a regular basis, and provide notices that adequately described reasons for budget reductions.
In a follow-up decision dated March 28, 2016, pursuant to the Ninth Circuit’s 2015 decision, the Federal District Court of Idaho stated:
IDHW’s ban [on disclosing budget methodology] prevents participants from challenging errors or effectively cross-examining [people who input participant data into the budget tool] who claim their assessments are accurate. Finally, the risk of error – either mathematical, clerical, or substantive, as discussed above – is substantial. The risks of erroneous deprivation outweigh the harm described above [copyright claims of software creator]; the
Matthews analysis compels a finding of a due process violation.
Of note, the Wisconsin Department of Health Services (DHS) has made some efforts toward transparency about at least one aspect of the IRIS budget creation process. In Wisconsin, a Long-term Care Functional Screen (LTCFS) is used to help DHS determine eligibility and budgets for IRIS participants. The DHS website provides a document that describes the original data inputs for the creation of the LTCFS, as well as testing done to ensure its reliability and validity.
Due Process versus Trade Secrets
K.W. ex rel. D.W. Armstrong is significant, because it contained a court order for a government agency to turn over its software methodology for public inspection, implying that the public interest in understanding automated decision-making outweighed the proprietary interest of the software creator.
Trade secrecy in decision-making software is arguably the biggest obstacle to attorneys making arguments based on algorithm bias: how can we argue bias when we don’t know what we’re dealing with? K.W. indicates that one option is to claim due process violation when individuals are not given meaningful opportunity to understand adverse decisions.
Section members: Get $50 off registration for the live webcast presentation Wispact Update 2021, courtesy of the Elder Law and Special Needs Section. The CLE program is 8:30 a.m. to 4:15 p.m. on Friday, April 30. For more information, see WisBar's Marketplace.
This article was originally published on the State Bar of Wisconsin’s Elder Law and Special Needs Blog. Visit the State Bar sections or the Elder Law and Special Needs Section webpages to learn more about the benefits of section membership.
Endnote
1 K.W. ex rel. D.W. v. Armstrong 789 F.3rd 962 (9th Cir. 2015).
