An invited presentation to the AI in Asia conference, Korea University Law School, Dec 16, 2016.
Synopsis
Computer Science teaches us that not all problems are computable. Some tasks do not have algorithmic solutions. Yet AI is premised on there being algorithms for thinking and human problem solving. This seems incredibly optimistic.
The point of ethical dilemmas like the Trolley Problem is that there is no one answer. The proper course of action will always be contestable because it depends on one’s philosophical frame. The old saw “reasonable people can disagree” characterises human affairs but it is unusual for algorithms to disagree.
Interesting human decision making tends to be unpredictable. Consider that court cases are commonly unpredictable – even by experts. There can be no algorithm for deciding a legal case, for there are always unexpected inputs and outputs; these are what make for legal precedents.
Some Self Driving Car makers have imagined having configurable ethics settings, to bias the machine’s decision making in a Trolley Car sort of scenario, towards either saving the driver or saving pedestrians. In my view, because life and death decisions can’t be algorithmically reliable, it is deeply unethical to consider such a feature.
As neural networks become more complicated, it will get more difficult to analyse their behavior. Already, deeply counter intuitive results are being seen (like the Carnegie Mellon researchers’ patterned goggles that fool facial recognition algorithms). When a neural network fails, how will we interrogate it? If a car accidentally kills someone, how will we ask it “what were you thinking?”.
There will be truly unpredictable failure modes in AI. The impossibility of computing all scenarios means that some robot behaviours will not be foreseeable in any detail. If the design process is unable to identify all failure modes and quantify the failure rates, it seems reckless to me to release robot cars on public roads.
We may need a new approach to pragmatic computability to arbitrate reasonable solutions to real world problems when we know that no algorithm exists. If a computer gets to the point of not knowing the correct course of action, what should we expect it to do?
Can an algorithm even recognise that it doesn’t know the answer?
Consider that it is not possible to ‘surprise’ an algorithm in the human sense of the word. When processing any given input, an algorithm will either:
- know exactly what to do with the input (because it has been programmed so) including rejecting input values that are detected by design to be invalid, or
- it will behave in an unexpected way.
So how can an AI self-diagnose what humans would sense as a state of confusion?
Just as we learned a great deal about human cognition from studying rare cases of brain damage, we might need a new branch of ‘AI pathology’, to learn systematically from mistakes. But this will require transparency amongst AI businesses that tend to be secretive given the commercial importance of their work.
Expectation setting
It’s early days for AI, so expectation setting is crucial.
For one thing, we’re still learning about learning. Watch closely when a parent is teaching a child to ride a bicycle. Beyond the very basics like “hold onto the handlebars with both hands” there isn’t much spoken instruction. Instead of explicitly teaching anything much at all, the parent instead lets the child learn to ride, providing protection and encouragement.
It seems likely that most complex social tasks are learned rather than expressly taught. The way humans do this is not yet understood.
Machine learning will be one of the greatest developments in the digital world, but let’s not underestimate the challenges, not oversimplify the regulatory paradigm shift that’s needed. We cannot simply transfer centuries old societal compacts and ethical norms into artificial intelligences, as if robots will behave just like us.