The growing influence that these black boxes have on our lives has led a team of researchers at MIT Media Lab to call for a new field of multi-disciplinary research they call "machine behaviour".
The new discipline draws on the expertise of biologists, economists, psychologists and other behavioural and social sciences to understand why algorithms make their decisions by looking at their actions rather than their inner mechanisms.
Lead researcher Iyad Rahwan draws a parallel between machine behaviour research and the field of ethology. While our understanding of the brain's mechanisms remains limited, by integrating behavioural psychology with neuroscience we can gain a fuller understanding of human behaviour.
"We could do the same thing with machines," Rahwan tells Techworld. "We just need to legitimise this type of study. And you don't have to be a computer scientist who understands internal mechanisms in order to state and discover facts about the behaviours of these systems."
Rahwan believes that this multi-disciplinary approach is essential to address the risks of algorithmic bias. He uses the example of hiring decisions that discriminate on the basis of race or sex.
"Who's an expert on whether a hiring decision is somehow racist, or somehow sexist? If you ask somebody on the street, they probably won't guess a computer scientist. They probably would say, maybe it's a sociologist, maybe it's a political scientist, or maybe it's a psychologist," says Rahwan.
"It is these people who have studied these phenomena as they are exhibited by humans, and what we're trying to promote is for them to do the same for machines. This is not to say that computer scientists can't be part of this kind of study. What we're saying is, the behavioural scientists can help enrich and complement the kinds of studies that computer scientists are doing. But it shouldn't be left to the computer scientists alone."
Developing a new field
Rahwan believes that machine intelligence will need to be recognised as a legitimate field of research by both computer scientists and experts from other disciplines before the different groups begin to collaborate.
It will also require supporting infrastructure. For example, economists studying pricing fixing among online retailers might need a toolkit that can simulate the behaviour of a consumer browsing their sites.
Rahwan predicts that the field will develop in academic studies that attract collaboration from independent experts and the tech companies that develop AI systems.
"Internally, many of these companies do hire behavioural scientists that are helping understand and design their systems better, but they're not independent third party scientists who would publish the results no matter what they were," he says.
"At the end of the day, they are in-house. They do collaborate with people outside the company, but there are very stringent checks on what kind of work they can publish, because if it had some kind of controversial findings that may make the company look bad."
Rahwan posits two major incentives for these companies to support the field of machine behaviour: gaining the trust of consumers and navigating future regulations.
"When the regulations come, you want there to be a well-developed way for companies to be able to demonstrate to the regulators that their systems are compliant," he says. "I think developing this capability of being able to interrogate your systems and understand them in order to demonstrate compliance is going to be important."