Oversight AI: AI to Control AI

Co-authored with Karthik Bharadwaj, cross-posted blog

The pervasiveness of AI may become one of humanity’s greatest achievements, having already changed and shaped our lives in many ways. We have seen the introduction of AI applications helping physicians identify cancer cells, farmers change the way food is grown, prevent online fraud and even provide us with dating recommendations. 

Through the exploration of AI, we had opened a door to a box of wondrous possibilities, however, this is a box that we have only just begun to understand – both for its marvels as well as unintended consequences.

Weighing the risks

Outcomes can be considered a benefit as long as they outweigh the potential risks they may pose. In order to adequately weigh such considerations, it is important to understand the consequences of a process’s components contributing to risk. In the case of AI, these components are primarily the data used for model development and the algorithm type selected. 

The risks of bias and lack of explainability; bias in data may propagate through the modeling process and result in discriminatory outcomes, while the lack of algorithm explainability may hamper the ability to measure the stability of AI recommendations. An example of this is the proprietary system that measures the performance of teachers in schools, which was ruled out by a federal judge as it was not possible to verify the accuracy of the results. 

Additional vulnerabilities to be considered are external risks such as adversarial attacks. The consequences of such adversarial attacks can be vividly illustrated when a group of researchers confused the AI of a self-driving car to mistake a stop sign to be a speed limit sign. 

Given these potential adverse outcomes, there have been ongoing endeavours by governments and technology companies alike to propose guidelines to regulate AI. Similarly, there has been growing research on AI safety and  AI explainability. These necessary efforts are promising, however, may take time to reach maturity. Should we concurrently explore  AI to monitor or control AI?

Smarter instruments to regulate smarter instruments

The world has been primarily focused on the development of AI in pursuit of commercial value. Moving forward we may need to explore a new category of AI that can have oversight on other AI programs, a supernanny AI, to ensure adherence to regulation and governance and that AI recommended decisions do not cause harm. 

A supernanny AI system will audit, govern and regulate the operational AI programs. It is aware of the goals and design specification of the AI system it governs. A supernanny AI can thereby control an operational AI malfunction or prevent significant deviations from the objective incorporated in the AI programs. A supernanny system audits the underlying data and algorithms of an operational AI system. Thereby, it can identify both data bias and if an algorithm deviates from its objectives. It aligns decisions of operational AI systems with AI ethical principles of trust, fairness, privacy, reliability and safety. It will be vigilant to identify adversarial attacks and take measures to stop it from having a major impact. 

An example of such an oversight AI would ensure self-driving cars to follow traffic laws and prevent adversarial attacks on them. Another would be a cybersecurity AI to ensure internet bots are prevented from getting into “raging wars” and also prevent the spread of disinformation from fake news.The oversight bots could monitor unauthorized access and prevent theft of information by bots. An interesting application of AI oversight in finance would be to prevent autonomous trading algorithms perturbing the financial markets by automatically shutting down the AI trades. 

These AI nannies have an advantage over their human counterparts – they can operate at the same scale and speed as, or more than, the AI agents they are tasked to monitor. It allows us to solve problems at a fraction of time and an ability to scale our AI regulation capability.

The vigilance of oversight AI could mean zero tolerance to deviation from objectives. A self-driving car slowing down, for the rider to enjoy the scenic view would be cautioned. We would be warned of a social network trying to get our personal information. Adaptation to world views would come with time for oversight AI and leveraging on when in doubt ‘Stop’ – mantra will help us educate it. Ultimately creating an AI-Human collaboration for monitoring, intervention, and review to help assure that the benefits indeed outweigh the risks.