By Ganes Kesari
These are exciting times for data science professionals. With the rapid advances being made in analytics, specifically around machine learning, deep learning and AI, researchers and practitioners around the world have started adapting these technologies for deep, human-impacting applications.
The Age of Advanced analytics
Social talk is replete with conversations that have shifted gears from self-driving cars to flying vehicles, smart machines to robots that are already collaborating quietly in our homes, and chatbots to conversational AI which is becoming omnipresent, contextual & indistinguishable from human response.
Many pride on the fact that, as a human race, we have managed to create real-time intelligence, one that hasn’t naturally evolved. With advances across disciplines, we have finally cracked several challenges in simulating human intelligence and are starting to surpass it in some areas.
The great analytics divide: Salvation or Sorcery?
Interestingly, this is where public opinion across the world also starts splitting into a clear divide. At the other end of this spectrum, we have people who question the very existence and intent of AI, with deep debates on not just the trustworthiness of advanced analytics, but also on its very utility.
At some level there is a sense of fear gripping most consumers. There is an eerie uncertainty around the optimistic spin being given to the areas of potential application, with questions raised on its feasibility, scale & impact.
There’s a connection between the advances that are made in technology and the sense of primitive fear people develop in response to it. — Don DeLillo
Rapid advances in technology have always been accompanied by an escalation of fear over the past decades. While technology earlier was complex, it was perfectly rational. With deep learning & AI, we can no longer claim this, since they now surpass the realm of human logic & understanding.
The Challenge of Analytics consumption
Lack of basic awareness is the biggest challenge facing the analytics discipline today, and this ranks higher than the ethical dilemmas around its adoption. As users increasingly lose grip around the understanding of a new technology, the noises around its perceived utility and questions of adoption get louder.
While large-scale adoption of advanced analytics by end consumers is still playing out incrementally, this is already a clear and present challenge for enterprises. Notwithstanding their million-dollar investments in data science to glean intelligent insights, businesses face huge resistance from within.
Low-code machine learning solution development reduces many roadblocks by decreasing the go-to market time of solutions to a great extent.
The biggest challenge in enterprise projects is not with model engineering or accuracy, but with the on-ground adoption of analytics applications and implementation of recommendations provided by these intelligent models. More so, when they end up counter-intuitive to industry heuristics & gut feel.
Models with high accuracy and low acceptance
Over the years of implementing advanced analytics engagements, we have come across way too many instances wherein, outstanding complex models (black-box) meet the engagement objective with exceptional accuracies, but fail to meet the human acceptance standards.
While black-box models like Neural networks bring about a significant jump in accuracies, this is often at the cost of losing explainability and human comprehension. Contrast this with the easiness in explaining a decision tree model with intuitive if-then-else conditions and simple thresholds.
However, in projects, the improvements in business benefits made possible by the complex, black-box models are too alluring to ignore. As data science practitioners, our responsibility is to bridge this divide, enable consumption of machine learning insights & gently push towards prescriptive actions.
If you can’t explain it simply, you don’t understand it well enough. — Albert Einstein
A Visual framework for Machine learning
While charts are more powerful in conveying information and can prove to be far superior to a table of numbers, a visual framework can be particularly effective in humanising the intelligence from machine learning.
Lets look at the 4 key elements of this visual framework which can promote easy comprehension and aid in demystifying advanced analytics models.
1. Information Design:
Visual story-telling of data is the foremost approach to present not just a table of numbers, but importantly the statistical results and interpretations of the algorithm results to arrive at prescriptive actions.
A standardised approach to information design with a user-centric approach, and by designing the right navigation workflow, pertinent representations and relevant visual design is the right place to get started on this journey.
2. Adaptive abstraction:
The most powerful way to gain insight into a system is by moving across the levels of abstraction. Data scientists and designers instinctively move up-and-down across different levels to glean insights & lay out the solution for users.
Its imperative to empower users with some fluidity, so that they can take in the bird’s eye view (abstract summaries), digest the ground-level detail and dynamically navigate across, adapted to context and user expertise.
Bret Victor’s ladder of abstraction is a useful reference, wherein he demonstrates steering around a problem by abstracting over time, algorithm and data. Applying this in a contextual, domain-driven way can demystify the analytical solution by shining light on the underlying design approach.
3. Model unravelling
Equally important in the journey of onboarding users is providing them a sneak peak into the model internals, albeit in a non-overwhelming way. While it still befuddles humanity on how algorithms like Neural networks learn or map data to the desired output, research is fast progressing in this area.
There are several early attempts at unpacking the internal sequence of steps in deep learning, particularly in areas such as classification and image recognition. Keeping the user safe from toxic statistical jargons, if we can unzip the model and enable simple traceability to the outputs, this can go a long way in making people appreciate the beauty of these black-box models.
4. User interactivity:
User interactivity can be that powerful glue in stitching together various elements of this framework. It enables a visual storytelling interface that promotes meaningful user-journeys across the levels of abstraction, to understand the salience and operation of a machine learning model.
By making all user interactions consistent, perceivable, learnable and predictable, the entire experience can be turned around from one which is doubt-inducing to something that can be meaningful & awe-inspiring.
Summary
While the data science and AI disciplines go through exciting and exhilarating advances, its important to keep the user’s expectations and experiences in perspective. This is very critical, since a sizeable segment of target AI users are fast getting alienated with deepening disconnect and distrust.
It doesn’t need a big dash of imagination in bridging this gap. Many of the enablers needed to build user trust and promote understanding are already available in our toolkit, and research in the field is quickly unraveling the rest.
What’s needed is an acknowledgement of this divide and a conscious effort to address it by adapting the above visual framework along with the constituent 4 key aspects, while implementing machine learning solutions.
Came across other striking examples from the realm of visualisation of machine learning? Please add them to the comments below.
This is a repost of https://medium.com/@kesari/
Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article.
Fantastic blog extremely good well enjoyed with the incredible informative content which surely activates the learners to gain enough knowledge. This, in turn, makes the readers explore themselves and involve deeply in the subject. Wish you to dispatch similar content successively in the future as well.