It is no secret that companies store huge masses of all kinds of data also known as the Big Data. Although a certain fraction is well-organised and thus easily accessible, at least half of the stored data is still unstructured. According to Veritas, some 52% (!) of all information stored and processed is in fact Dark Data. In another words, Dark Data is a type of untagged and untapped data that is found in data repositories and has not been analysed or processed. It is a subset of the Big Data but differs in how it is mostly neglected by business and IT administrators in terms of its value. Dark data is also sometimes referred to as Dusty Data. These can be all sorts of files:
• Log Files
• Raw Survey Data
• Email Correspondences
• Account Information
• Notes or Presentations
• Old Versions of Relevant Documents, etc.
Whereas it is commonly believed that data generally lead to improved decision making, many business leaders are unaware of the fact that the Dark Data can be more of a hindrance than help. Apart from its uselessness, it might actually turn out to be damaging for a business. First of all, storing vast amounts of data doesn’t come cheap. Companies tend to pay way more cloud storage space than they need, which inevitably causes unnecessary costs. Secondly, in the era of ever tightening data regulations, many companies might get into trouble should they be accused of neglecting sensitive personal information. The trend seems even more alarming considering the fact that hackers could potentially sip through the accumulated ‘databergs’ and get hold of sensitive material undetected.
Yet the Dark Data only presents a problem as long as it is not suitably tackled. The good news is that with the rise of AI-based analytical tools it might become easier for businesses to decipher the hoarded tons of information and unlock the potential that lies within. Of course, this is easier said than done: While larger companies tend to be better prepared to handle Dark Data, it is not always the case for SMEs. However, modern analytical solutions are gradually emerging which should also grant an opportunity for smaller businesses to structure their data and thus better comprehend the market. At the end of the day, it is all about the competitive advantage: The ones who will succeed in making meaningful connections out of their untapped data possessions will be the winners; the rest will be lagging behind.
https://www.techopedia.com/definition/29373/dark-data
https:// disruptionhub.com/drowning-dark-data/
https://www.veritas.com/news-releases/2016-03-15-veritas-global-databerg-report-finds-85-percent-of-stored-data
https:// disruptionhub.com/glance-dark-data/
Nice read. Indeed, by leveraging AI the heaps of untagged data can be remedied and therefore prevent any problems or create competitive advantage in the future. The high-performance Customer Information Management tools automate and accelerate processes, connecting data sets for clarity and insight. Subsequently, businesses can start to explore the vast opportunities which lie within the data.
Thanks for the valuable insight on this topic. With an increasing trend towards getting “information” from your “data”, this issue will take more and more space on business magazines for sure. In my opinion, the concept of dark data and all the data that is unused or unutilized will lead to a boom of startups specialized in generating information from databases of large enterprises. By having access to the database of several big players, they can generate even more information than a player can do on its own. So this can also be an opportunity for students like us to start with our own business. Afterall we will be able to mine data and make information out of it in few months 🙂