What if there existed a technology platform that could reveal insights from large amount of unstructured data? Guess what, it already exists. IBM introduced Watson, an artificial intelligence technology that can actually learn to extract insights out of huge amounts of unstructured data.
80% of today’s data is unstructured. This unstructured data consists of for example news articles, research reports, social media posts and information system data. Watson can extract the key elements of this data in minutes.
But how does this work? Watson uses natural language processing to understand the grammar and context of the data. You even can ask Watson questions, these questions are evaluated by Watson thoroughly until every possible meaning is covered.
Watson is like an over motivated student. It is eager to learn a subject, it will go through all the data that is uploaded about a certain subject. The next step is training. Watson can be trained by asking questions and revealing the answers afterwards. Finally, Watson can answer every possible question about the topic by searching through millions of documents. The answers, usually thousands of them, are ranked by Watson using a scoring algorithm to rate the quality of these answers. You end up with a structured overview of evidence.
But the question is, does this make humans unnecessary in the process of analysing big data? I do not think so because all the data that is used is provided and developed by humans. Watson will always be subject to human thinking patterns. Is the technology intelligent, yes, but only because of the intelligent people behind the technology. However, less people are needed in the process because Watson is somewhat quicker than us humans.
Do you think Watson will take over big data analysis?
Sources
https://www.youtube.com/watch?v=_Xcmh1LQB9I
Hello Jonne,
Thank you for your interesting post. To answer the question if I believe that Watson will take over big data analysis completely, I believe not. Of course Watson can analyze millions of pages structured and unstructured data faster than a human mind (or 1000 humans…). However the data presented is often incomplete and in many cases essential information is not present. In these cases there will always be a need for human action. I agree with your opinion that Watson or other artificial intelligence technology could take over a big part of data analysis (when the information is available and complete) and less people are needed in the future .