Organizations are increasingly turning to machine-learning models to allocate limited resources or opportunities, such as screening resumes for job interviews or ranking kidney transplant patients by survival likelihood. To ensure fairness, users often employ techniques to reduce bias in these models, such as adjusting features or calibrating scores. However, researchers from MIT and Northeastern University find that these methods may not be enough to address structural injustices and uncertainties. They propose introducing structured randomization into the model’s decisions to improve fairness, particularly in situations involving uncertainty or consistent negative decisions for a particular group.
The researchers present a framework for incorporating a specific amount of randomization into a model’s decisions by utilizing a weighted lottery for resource allocation. This customizable method can enhance fairness without compromising the model’s efficiency or accuracy. They suggest that as the use of algorithms for social allocations of resources increases, the inherent uncertainties in these scores can be magnified, making fairness necessitate some level of randomization in decision-making. This approach, according to lead author Shomik Jain, acknowledges the uncertainties in assigning opportunities and prevents one worthy individual or group from always being denied scarce resources.
The researchers build on previous work that highlighted the harms of deterministic systems at scale, showing that using machine-learning models for deterministic resource allocation can amplify inequalities in training data, reinforcing bias and systemic inequality. Through their analysis, they found that randomization aligns with fairness demands both on a systemic and individual level, as outlined by philosopher John Broome’s perspectives on using lotteries to allocate scarce resources. Recognizing that fairness requires respecting individual claims to resources based on merit, deservingness, or need, the researchers explore when randomization can enhance fairness through structured allocation.
In decision-making processes where determinism could lead to systemic exclusion or exacerbation of inequality patterns, randomization can help avoid repeated mistakes and ensure fair outcomes for individuals. The researchers advocate using a weighted lottery system to adjust the level of randomization based on the uncertainty present in the model’s decision-making process. By leveraging statistical uncertainty quantification methods, one can tailor the amount of randomization to specific situations, such as in the case of kidney allocation where projected lifespan is a deeply uncertain factor. Calibrated randomization can lead to fairer outcomes without a significant impact on the model’s utility or effectiveness.
While the researchers acknowledge that randomizing decisions may not be suitable for all scenarios, such as in criminal justice contexts where it could be detrimental, they emphasize the potential benefits in areas like college admissions. They intend to explore additional use-cases in future research and study how randomization can influence factors such as competition or pricing, as well as enhance the resilience of machine-learning models. By providing randomization as a tool for improving fairness, the researchers aim to spark further discussions among stakeholders involved in resource allocation to determine the appropriate level of randomization needed for particular situations, acknowledging that the decision-making process itself is another area of research interest.