Analytics, Augmentation, Automation, Adaptation & Autonomous
We need to acknowledge the difference between data science, machine learning and artificial intelligence. By understanding the different we can clearly demonstrate the path from analytics to adaptive intelligence systems. While data analytics can deliver excellent insight into the underlying business dynamics, true artificial intelligence will only be demonstrated when systems can leverage the full suite of techniques and tools of AI within a framework that supports the application of augmentation, automation and finally adaptive methods.
To explore this in more detail, let us try to describe what we mean by the four ‘A’s, however, there needs to be some caution, as we note that each one has some overlap with adjacent layers of the pyramid. The top of the pyramid is Analytics, and is equivalent to the tip of the iceberg. The bottom of the pyramid, is adaptive systems, this being the most sophisticated of the four ‘A’s and the most complex to deliver, but with the most significant benefits. Augmentation and Automation are stepping stones from Analytics to Adaptive Systems, but necessary steps that each can be demonstrated to show benefit to the business while providing the opportunity for the business to gain confidence of the systems put in place.
What we are seeing across many industries, is a focus on leveraging big data platforms and data lakes by applying just data science techniques in the form of data analytics. It seems that at the moment, few if any companies realise the power of a platform that not only delivered advanced data analytics and visualisation, but can help them on the journey to produce fully automated and adaptive intelligence systems. Such systems could support entire business processes, freeing up staff to focus on more complex tasks, leaving the automated systems to cover the high volume simpler tasks.
Hopefully some of the companies that are currently building the tools and frameworks that are currently available are planning to deliver more advanced platforms in the future, However, unfortunately at the moment it appears that companies are not producing the platforms and frameworks that are needed to support the development of all four ‘A’s. This seems to be a huge missed opportunity, and should be avoided if we want to prevent the AI field stalling again (avoiding the third AI Winter).
What needs to happen next is to deliver platforms that can augment manual workflow by providing semi-automated systems that support business process and enable the subject matter experts to focus on the more involved and complex elements of the business process. Moving this platform forward to a point where the majority of the process is fully automated would add significant benefit to any business workflow. The next layer of the pyramid, is adaptation, and this is where we get fully enabled intelligent systems that can adapt to changes in the underlying data and continue to perform at the level of performance when the system was initially introduced. The bottom layer of the pyramid is that of autonomous, where the application can perform in any situation and condition without the need for human intervention.
I would like to see this pyramid referenced for the development of machine learning frameworks and artificial intelligence platforms, to guide the type of capabilities being build and deployed.
We should not just strive to build data analytics, but produce intelligent systems that can augment and automate business process in a way that adapts to changes in the data space over time providing systems that continue to deliver performant automation until we reach the level of full autonomous system that operate completely independently.
Analytics – Enabling Business Decisions
Business Intelligence Reporting has always been about delivering business users and the C-suite with the reporting and intelligence that they can use to make decisions and run the day to day operations of the business as well as look at more strategic thinking and forecasting. The technologies of big data and data science have completely turned the area of reporting and business intelligence into a capability for advanced forecasting and prediction, that have made the initial focus for AI, machine learning and data science be grounded in the data analytics world.
While this is a great place to start, this is only the tip of the iceberg for the field of Artificial Intelligence. The analytics driven forecasting and predictive decision making gives a preview of how transformative these techniques can be for the entire business, its operations and strategic planning.
Defining Augmentation – Making the Leap From Analytics to Augmentation
Following on from the previous introduction article on the Four ‘A’s Pyramid Framework, we need to determine how we will make the leap from simple analytics to augmentation.
The combination of big data technologies with highly parallel computing power has enabled huge advances is data science, analytics and machine learning. Particularly giving us the ability to better visualise and understand the data being analysed.
Leveraging these advanced analytics including clustering and predictive algorithms on the surface appears to be very easy. There are many platforms now that make the task of training and deploying a predictive model relatively easy. However, while this maybe initially true, this simple setup doesn’t consider a number of important factors that will deliver robust and resilient intelligent systems longer term.
Lets consider the definition of Augmentation.
Augmentation is about providing the capability to support human activity via computational methods.
This maybe by providing visualisation of information and insight though clustering or ultimately via regression and classification to predict the correct outcome for a given application. Augmentation has the ability to take on the simple tasks previously done by a human, freeing the human to perform the more interesting or complex tasks. There are wider social and workforce implications with this, but they will be covered in other articles. For this article we will focus on the technical aspects.
There are gaps between the platforms and frameworks that are currently available and what is actually needed to provide smart, robust and resilient systems for augmentation. So what is missing?
Well what we need to understand is that training a predictive model is not a one off task and the data will inevitably change over time and will vary to what the model was trained on. This essentially has the effect of reducing the performance (accuracy) of the model over time. One approach to solve this is obviously to continually train the model, which does make sense and has the potential to deal with varying data over time. But this itself brings its own set of challenges. Including how do we select the right sample of data to use to train the model. How do we prevent localised skews or abnormalities in the data. How do we ensure that infrequent events are represented and can be part of the generalised model.
In addition we must factor in confidence levels from the model to determine if a specific prediction should be fully automated or needs to be reviewed by the user. The specifics of this will of course vary from application to application, but a standard way to perform this would be useful.
Also another key area that appears to be missing from the current platforms and frameworks is a standard way to feedback from the human as part of this close interplay between the user and the automation that underpins the augmentation. When the human flags a mis-classification, how does the machine learning model factor this into the re-training of the model.
Producing a system that can make accurate predictions only takes us part of the way to delivering a system that can augment tasks successfully over the long term. Any machine learning platform will need to provide algorithms and solutions to the above identified gaps before we can produce robust and resilient intelligent system.
What needs to happen next is to deliver platforms that can augment manual workflow by providing semi-automated systems that support business process and enable the subject matter experts to focus on the more involved and complex elements of the business
While there is a lot of excitement and optimism of what we can achieve with machine learning algorithms and techniques, we need the platforms and integration layers to facilitate a number of capabilities to support augmentation. Identifying these missing capabilities is the first step towards an intelligent system. The next is extending the platforms to deliver these capabilities.
Embracing Automation – From Augmentation to Full Automation
Following on from the previous articles on the Four ‘A’s Pyramid Framework, we need to determine how we will move from augmentation to full automation.
This should be a simple move, but in reality it is a very complicated process. Any business transformation is. But a process that involves the replacement of a complete process with a fully automated substitute is going to come with added challenges, not least resistance from those currently performing the process.
This is more than just a technical challenge, this is a socio-economic issue, that will require a number of polices, procedures, planning, preparation and persistence to overcome.
The one of the main challenges I see is building in the check and balances to ensure that the system is still inline with the environment. Over time, the environment, and thus the data that is being input into a system can change, shifting the shape and profile, new situations can occur that the system hasn’t seen before. How can we ensure that the system will behave correctly with these new unseen situations.
With Augmentation, we had the human in place to spot these abnormalities in the data and results and able to correct the outcome. With full automation we need to make sure the system will work efficiently both now and in the longer term. But now do we do that?
We have a number of strategies available to us. The obvious one is to constantly retrain the system on the most recent information / data. So as small changes in the environment happen, the system can learn them and deal with them properly. This is a good approach and will cope with most of the changes in the environment that are typically gradual over time. However, what about the “black swan” type events, that come out of the blue? Or up stream system failures that cause our input data to be significantly skewed.
Well we can build pre-processing on the data that checks for significant changes in the shape and profile of the data and raises an alert. But what would happen to that alert? How is this handled?
Well yes, even a fully automated system would need to have an over-ride, a way for us to manually correct / check things if we know there might be a problem.
However, this raises a problem, if humans are taken out of the loop for the majority of the work / time, over a long period of time we might loose our expertise and may struggle to know what to do ourselves.
This would then suggest, that we only automate the simple more straight forward work, that is easy to understand, less likely for an automated system to go wrong, but if is does, we would still be able to resolve it ourselves.
Automation has always been a part of our environment, however, it is only now we can see it happening in full force, impacting so many so quickly, that it is becoming a scary proposition for many.
While this maybe a difficult time, needing us all to adapt to the changes such automation brings, we should welcome the improvements to our lives and see the positive impacts as well as the difficult changes.
Adaptation – Keeping Automation Performing
Here we leverage the benefits of user feedback to corrections to re-train the system to improve performance in the future. However, this is only part of the story. Ongoing data changes, drift, creep can cause reduced performance over time.
Plus typically the business is not static, with new scenarios, new data fields, new situations all ensure that any static system will eventually break if it is not made aware of this new information.
Adaptation is thus fundamentally important to ensure the ongoing long term performant operation of any machine learning application or system.
For small changes, this is easy to factor into the overall design of the system, to do frequent retraining, either online, or offline. But what if changes are much more impactful, how should these types of changes be dealt with?
Autonomous – Building Robust and Independent Applications
Completely independent, without the need for the human oversight or monitoring, regardless of the situation or environmental conditions. The system can cope with any and all situations, both “normal” and “extra-ordinary”.
Autonomous systems will ultimately cope with everything it experiences, and result in a safe and acceptable result.
This is our last layer in the pyramid, and the most difficult to achieve, but this is the ultimate goal of AI powered systems, to fully take on the responsibilities of a specific business process or workflow and deal with any exceptions as well as the happy path.
Footnote
While typically the ordering of a pyramid would have the first levels on the bottom, with the later more complex levels towards the top, here I wanted to highlight two things by ordering A’s in the opposite order. Firstly to show that while alot of focus is on analytics at the moment, it is the tip of the iceberg, and secondly, the later levels, automation and autonomy are more involved and require alot of work to deliver, shown by the larger parts of the pyramid.