These Algorithms Are Judging You
Photo of Upstart’s Paul Gu via The New York Times
If algorithms are used for everything from finding missing pets to battling drought, why shouldn’t they also be used to determine someone’s character? That’s the question posed by several startups leaning on data analytics to develop formulas that effectively judge people in various capacities in ways that are – theoretically at least – less biased than a human.
“We all have biases about how we hire and promote,” Dan Beck, head of technology strategy at Workday told The New York Times in a story on the trend. “If you can leverage data to overcome that, great.”
As the Times describes, Workday, creator of cloud-based personnel software, has a new product that “looks at 45 employee performance factors, including how long a person has held a position and how well the person has done. It predicts whether a person is likely to quit and suggests appropriate things, like a new job or a transfer, that could make this kind of person stay.”
The idea is that by looking at data surrounding a large sector of the existing workforce and analyzing it for trends, employers can isolate the qualities in potential employees that determine their success. It offers a full spectrum view as opposed to examining only the traits of the most successful employees, resulting in information that is as preventative as it is predictive.
Others have found great success using algorithms to make judgments on the likelihood of success for specific businesses. Thomas Thurston of Growth Science, for example, uses data to predict if innovations will survive or fail through proprietary algorithms that have helped guide billions of dollars in investments. (Thomas was also a speaker at FTF: Conference 2015 – you can watch video from his panel here.)
Judging individuals’ character can be seen as the next logical outgrowth.
And these personality-based algorithms are not only used for the purpose of hiring or promoting. ZestFinance bases their decisions to loan to subprime borrowers using data as idiosyncratic as whether or not they’ve abandoned a prepaid wireless number. Upstart loans money based in part on individuals’ SAT scores and college GPA.
While all of this may make these processes seem more fair and less beholden to unavoidable human favoritism, there is one giant caveat: as the Times‘ Quentin Hardy pointed out, “Algorithms do not fall from the sky. Algorithms are written by human beings.”
So algorithms may get companies much closer to the truth about someone’s character than a typical interview process or screening, but it’s still a very specific version of the truth that is, by design, at least a little bit biased and potentially flawed. Perhaps most tellingly, the Times ends their exploration of the topic with a footnote that since he had dropped out of Yale, Upstart co-founder Paul Gu would not have qualified for an Upstart loan using the company’s initial algorithm.
