Is recruitment AI discriminating against job seekers?

It has recently been revealed that Amazon developed an AI hiring tool that appears to have delivered systematically discriminatory results.

The objective of the programme was to create an algorithm that would be able to identify the best candidates from a stack of CVs and that would track patterns in order to deliver better recommendations over time. 

Because the majority of CVs supplied over the previous 10-year period had been from men the algorithm began to optimise itself in their favour. It reportedly even downgraded graduates of all-female colleges and applications that featured the word “women’s”.

Lessons learnt

The issue was spotted, addressed and the programme formally ended in 2017 but it did raise the issue that the AI is only ever as good as its data. If populations are excluded from the dataset, or are heavily weighted towards certain outcomes, then the algorithm will continue the trendlines that it has been provided with – or average the subset out with the rest of the group. What’s worse, as seen here, is when machine learning algorithms lean into existing inequality and exaggerate it.

And given that AI is steadily advancing in HR, with AI expected to automate 16% of HR jobs in the next decade, it’s important issue.

The optimal end goal is of course for organisations to achieve the benefits of blind testing: getting a sense of precisely how well the individual will perform in a role, while discarding irrelevant details. 

Amazon’s dilemma presents a cautionary tale, underlining the need for human operators to oversee algorithms – and to be able to audit precisely why they do what they do (which may be difficult to achieve given the complex nature of the calculations at hand).