Big Data is the unexpected resource bonanza of the current century. Moore’s Law driven advances in computing power, the rise of cheap storage and advances in algorithm design have enabled the capture, storage, and processing of many types of data previously that were unavailable for use in computing systems. Documents, email, text messages, audio files, and images are now able to transform into a usable digital format for use by analysis systems, especially artificial intelligence. The AI systems can scan massive amounts of data and find both patterns and anomalies that were previously unthinkable and do so in a timeframe that was unimaginable. While most of the uses of Big Data have been coupled with AI/machine learning algorithms so companies and understand their customer's choices and improve their overall experience (think about recommendation engines, chatbots, navigation apps and digital assistants among others) there are uses that are truly industry transforming.

Jim Sinur, Ed PetersJim SinurDecember 21, 2020Jim Sinur, Big DataComment

AI & Big Data; Better Together

Big data, unstructured or structured, fast or slow, in multiple contexts or one is a beast to manage. Big data is growing fast fueled by the democratization of data and the IoT environment. Often organizations simply control what they know they get results from and then store the rest for future leverage. In fact, most organizations use less than 20% of their data, leaving the remaining 80%, and the insights it contains, to be left outside to the operational and decision-making processes. Imagine if you used only 20% of any service you paid for every month and ignored the other 80%! This is exactly what we are doing with data. Fortunately, there is hope as this is where Big Data can start to rely on AI and engage in a “cycle of leverage”. Presently, the interaction between AI and Big Data is in the early stages, and organizations are discovering helpful methods, techniques, and technologies to achieve meaningful results. Typically these efforts are neither architected nor managed holistically. Our work has shown there is an emerging “Cycle of Big Data” that we and would like to describe and share with you where we see AI can help.

Jim SinurJim SinurSeptember 25, 2020Jim Sinur, ai, Big Data, Big Data Cycle, Data Management, Pattern ManagementComment

Editorial