System helps spot bias in algorithms
There’s no question that algorithms can be biased, producing results that reflect the creator’s preconceived opinions. But how do you reliably detect signs of that bias? Carnegie Mellon researchers can help. They’ve developed a system that tests algorithms to see how much influence a given variable has over the outcome, giving you a sense of where bias exists. It could reveal when a credit score system is giving any weight to racial discrimination, or catch simple mistakes that put too much emphasis on a particular factor.
If the system finds its way into regular service, it could provide greater transparency all around. Companies and institutions could use it to conduct audits and spot flaws that would otherwise go unnoticed. There’s even a chance that you could use the system yourself — in a credit check, you could understand just how important it is to pay your bills on time. The system only works if the algorithm’s gatekeepers offer access in the first place, but it could make all the difference if it holds someone accountable when they try to rig data.
(23)
