The past buzz words “big data” and “machine learning” are incredibly powerful now and have immense future potential. We treat “big data” and “machine learning” as connected activities. People have been talking about the need for more ‘analysis’ and insight in big data, which is obviously important, because we’ve been in the ‘collection’ phase with big data until now. But the innovation in the big data world that I’m most excited about is the ‘prediction’ phase — the ability to process the information we’ve collected, learn patterns, and predict unknowns based on what we’ve already seen.
Let’s face it, big data is beyond our abilities as humans (and that of most of today’s analytic platforms). But coupling it with machine learning can deliver the Holy Grail for many organisations, in the form of accelerated value from their data investment.
Machine learning is to big data as human learning is to life experience: We interpolate and extrapolate from past experiences to deal with unfamiliar situations. Machine learning with big data will duplicate this behavior, at massive scales.
Big data represents the aggregate experience of the organisation – composed of, among others, financial, social and field data. As such, it is a strategic asset.
In fact, the ability to realise value from big data is one of the most important ways in which exponential organisations are gaining their disproportionate impact compared to their peers. To cash in on this currency of the new economy, ambitious organisations view big data as a tremendous opportunity to discover insights that can lead to better and faster business decisions. Accordingly, they focus on making analytics a core competency across the enterprise.
Where business intelligence before was about past aggregates (“How many red shoes have we sold in Kentucky?”), it will now demand predictive insights (“How many red shoes will we sell in Kentucky?”). An important implication of this is that machine learning will not be an activity in and of itself … it will be a property of every application. There won’t be a standalone function, “Hey, let’s use that tool to predict.”
Those with maturing analytic postures go one step further, by accelerating the value they gain from data assets through advanced analytics, including machine learning (ML). Together, big data and ML are ‘accelerating technologies’ that enable organisations to rapidly innovate and disrupt markets.
Only with ML can companies truly tap into their rich vein of experience and mine it to automatically discover insights and generate models to take advantage of all the data they are capturing. ML can, for example, help a company learn from past behaviour and predict behaviour of new customers, segment consumer behaviour in an optimised, market-friendly way (e.g. customer lifestyles modelled from geo-location data on cellphones), or conduct crowd simulation models where each customer’s response to a reward is modelled.
In time (very quickly in an age of accelerating change), companies that shun ML will fall behind. Each customer, partner or supplier response or non-response, transaction, defection, credit default and complaint provides an experience to learn from. While earlier paradigm shifts were sparked by steam engines, fossil fuels, electrical power, computers and the Internet, we are currently experiencing a boom driven by big data.
Gratifyingly, the market is taking the opportunity seriously. A recent survey reveals that 41% of organisations are using ML, 36% are experimenting with it, and 16% are considering using it.
Still, there is still some mystique and confusion surrounding it. Many software vendors claim they do predictive analytics, deep learning, prescriptive analytics and ML, when in fact they do no such thing. Machine learning, it should be noted, is the modern science of finding patterns and – as said above – making predictions, from data based on work in multivariate statistics, data mining, pattern recognition and predictive analytics.
Phew. But it’s not that scary, or it shouldn’t be. It’s no longer good enough for ML to provide the ability to achieve the most accurate, fastest and most scalable predictive insights. For it to make a difference it has to be delivered in a smarter, more usable form, enabling not only the data scientists who have PhDs but also the business users who tap into the technology.
Interestingly, ML is a key enabler of cognitive computing, or IT systems that can sense, comprehend and act. Leveraging computer vision, natural language processing and inference engines, cognitive systems enable more natural interactions with the environment, people and data. Which basically means ML is fixing itself and making itself into an indispensable business tool.
But machines, however smart and natural, don’t solve real, complex business problems. Humans do. An over-reliance on data can hinder innovation. To overcome this potential limitation, companies should establish a level of collaboration between humans and computers.
Through this collaboration, entirely new data associations can be discovered – ones that almost certainly could not have been made by humans alone. Machines can compute with incredible accuracy and scale, and can consistently get better at doing so. Humans complement this ability as they excel at thinking creatively and in context, so they can question and improve the conclusions of the intelligent software.
Indicating that this collaboration is seen as an important path forward, 78% of survey respondents stated that successful businesses will manage employees alongside intelligent machines.
So are you ready to get a little psychic? Whether you’re ready to adopt ML can depend on your organisation’s analytics maturity level, culture and the availability of technology and talent needed to support ML.
Once the time is right to begin exploring ML, you should develop a comprehensive understanding of your organisation’s software intelligence. Then, you should begin to invest in educating employees, hiring technical experts and encouraging use of tools like Python and other open source libraries.
Alternatively, if a company is experienced with analytics and pursues a lot of R&D initiatives, ML would be an excellent tool for them to use to fast-track innovation. These companies should begin by identifying their data assets, leveraging new ones, and starting to explore the data they already have in search of hidden insights. They will start small and add on until machine learning is prevalent throughout the enterprise.
Happy exploring!
Please feel free to leave a comment or question regarding this topic.
[divider scroll_text=”Back To Top”]