When it comes to AI singularity, we are still too far from achieving general AI to let this concern keep the masses up at night. The application of AI in ways that are harmful to a fair and democratic society is a bigger problem than solely the performance and behavior of AI algorithms. Most research and new technologies could be used in ways they were initially not intended to and finally harm our societies. With that said, there is certainly the need for regulation and worldwide agreement to prevent the use of AI in unethical and unwanted ways. Such regulations are already being discussed and will likely be implemented soon, for example, in the form of the proposed EU AI Act.

To understand why uninformed and careless use of AI systems poses a risk, let us consider the following, hypothetical, scenario: You have created a decision support system that is designed to help healthcare professionals to identify possible signals of addiction among patients. Your system is becoming an appreciated tool and is adopted within the healthcare industry. After some time, you learn that a patient was wrongfully accused of being an addict all due to the incorrect prediction of your decision support system. You knew all along that your system was not always correct in its prediction, as is the nature of a predictive model, but the system was meant only to support the healthcare professional, and not to replace his or her judgement. This hypothetical example closely resembles an actual situation that did occur within the healthcare industry. One of the problems, in this case, was that the system was not used in the way it was intended to, and somewhere along the way, it transformed from being a decision support system to making the final decision on behalf of the healthcare professionals.

To take this addiction scenario further, let us consider the reasons for the incorrect prediction. In our thought example, the patient got high addiction scores due to the historical prescriptions being given by several different doctors, indicating that the patient wasn’t seeing one doctor solely, a behaviour often found among addicts. In reality, several of these prescriptions were given by veterinaries for her ill pets. The AI model did not distinguish her prescriptions from her pets’ as they were all prescribed to her. If the addiction score had come with an explanation of what influenced the score, the healthcare professional would have had a chance to discover that the strongest driver of the high score was caused by a data issue. Therefore, the model should not have been trusted in this particular case.
To help ensure that AI systems are used the way they are intended and to highlight the limitation of these, a proposed solution is to define a content declaration attached to every AI model. This would contain information about how the model was created, which data was used, the expected accuracy of the model, and its intended usage.

My point is that if the healthcare professionals had been better informed on the intended usage of the model, its limitations, and how it worked, this case of unfair and inaccurate treatment of the patient would have been prevented. We grow up learning that the world isn’t a fair place but that shouldn’t stop us from trying to make it one. AI technology has a great potential to improve the lives of many which is why it is important to use it responsibly.