While the algorithms, infrastructure, and programming are all crucial parts of machine learning and AI, we cannot forget about, most probably, the most vital component - adoption.
Source: "The State of Machine Learning Adoption in the Enterprise", O'Reilly, 2018
One recent research conducted by O'Reilly shows that the stage of putting models to production is very initial for the vast majority of the surveyed respondents. The reason for that is predominantly culture and organization. Alike in many other fields, in the data science teams, we now use the title that didn't even exist 5 to 10 years ago. Despite the expanding presence of AI-powered technologies in our daily lives, there is still much distrust in the algorithms - we people tend to worry about what we do not entirely understand. The natural need for understanding brought to the point when we need to build trust around the analytics solutions to let them prosper.
The above concerns are the fundament for the explainable artificial intelligence (XAI) and further development in this fantastic field. One of the examples where understandable model outputs are crucial is hiring. Not all of the HR digital systems address the issues of discrimination and cannot ensure the hiring process fairness.
Source: www.nytimes.com
[H]iring could become faster and less expensive, and […] lead recruiters to more highly skilled people who are better matches for their companies. Another potential result: a more diverse workplace. The software relies on data to surface candidates from a wide variety of places and match their skills to the job requirements, free of human biases. - Miller (2015)
Imagine Susan or James, the analytics leaders, trying to convince business stakeholders that the models' results are meaningful. The fundamental "Why?" question very often pose a threat to the machine learning models their team builds. The ability to explain models' results, even those of very accurate predictions, plays a significant role in their adoption. There is plenty of skepticism and sometimes lacking trust in the models that are being produced by different analytical teams and data science hubs. And the doubts are very often rational because people without a required technical or analytical background cannot understand why the model shows the results it shows. The decreased confidence can impede taking business decisions in a data-driven way, event if the provided insights are sound, which in many cases can lead to lost opportunities.
Some researchers argued the difference between description and justification - by making not only how visible to users, but also the why. With this in mind, there are some key concepts I would love to elaborate on and discuss further. The field of explanatory artificial intelligence is flourishing, and that caused the rise in the number of researches. The explanations are essential to ensure algorithmic integrity and fairness, identify potential bias/problems in training data, and ensure that the algorithms perform as assumed.
Definitional debate
There are three major components that I would like to highlight: interpretability, explainability, completeness. Interpretability is loosely defined as the science of comprehending what a model did or might have done. But we take the stance that the interpretability alone is insufficient. We, humans, need a clear explanation to trust black-box algorithms - the rules that generate insights about the roots of their decisions. We often hear interpretability and explainability being used interchangeably, but there are critical regulatory reasons to distinguish between them. By default, explainable models are interpretable, but this is not necessarily true in reverse. The other notion which is worth underlining is completeness - the goal is to describe the operation of a model accurately. Exposure of all the mathematical operations and parameters in the model or method when explaining a computer program such as a convolutional neural network can be treated as its complete explanation.
Ethical concerns
Multiple questions can arise when building interpretable systems concerning ethical aspects of it. Some of them are:
When is it dishonest to manipulate an interpretation to persuade users better?
How do we find a sweet spot between transparency & ethics and the appetite for interpretability?
Some researchers believe that it is fundamentally unethical to present a simplified description of a complex system to increase trust if users cannot understand the simplified description's limitations. The worst scenario is if the explanation is optimized to hide undesirable attributes of the system. Rather than providing only simple descriptions, systems should allow for descriptions with greater detail and completeness at the possible cost of interpretability. There is always a tradeoff between interpretability and completeness - we are to find the right balance.
Conclusion
Securing the fairness of machine learning systems is a human-in-the-loop process and crucial for the adoption process. People are in the center of the entire development cycle - it relies on developers, users, and the broad public to identify integrity problems and make improvements.
We train machine learning algorithms based on data from the past. The decisions made in the past may have been biased and discriminatory. People would be more likely to trust the underlying ML systems when they believe the explanation that follows the model.
But software is not free of human influence. Algorithms are written and maintained by people, and machine learning algorithms adjust what they do based on people’s behavior. As a result […] algorithms can reinforce human prejudices. - Miller (2015)
The adoption will absolutely benefit from increased confidence that the algorithms are becoming less black- and more glass-box. The post you have just read is just a highlight of what I consider the most interesting conceptually from a couple of extraordinary papers. There are plenty of materials that can help you keep up with the XAI domain. I encourage you to learn more as these are us who are pushing the adoption to the next level!
In my upcoming posts, I am going to introduce one of the tools that handle bias detection and models' explanations. I will show how to train an example model using TensorFlow, deploy with TensorFlow Serve on Docker, and later explain. Until then you can take a look at the hands-on guide for Docker containers.
See you there :)
References and inspirations:
Dodge, Jonathan et al. "Explaining Models." Proceedings of the 24th International Conference on Intelligent User Interfaces (2019): n. pag. Crossref. Web.
Leilani H. Gilpin and David Bau and Ben Z. Yuan and Ayesha Bajwa and Michael Specter and Lalana Kagal, "Explaining Explanations: An Overview of Interpretability of Machine Learning"
CS 294: Fairness in Machine Learning, Moritz Hardt
Comments