Descriptive Analytics, the lost idea in the current AI hype

Descriptive Analytics

In the current wave of excitement around AI and generative technologies, the value of descriptive analytics as a foundation for predictive modeling is often overlooked. Descriptive analytics goes beyond traditional exploratory data analysis; it leverages statistical and machine learning methods to systematically summarize, interpret, and explain the essential patterns and concepts within your business data. This process provides critical insights that should inform and guide subsequent predictive modeling efforts.

As datasets grow in size, summarizing them becomes increasingly essential. Traditional visual inspection methods, effective for small datasets, quickly become impractical as data scales to thousands or millions of rows. To extract meaningful insights, it is crucial to identify the most informative subsets of data, distinguishing them from less relevant portions. Both the optimization and machine learning communities have developed innovative techniques to efficiently summarize and interpret large, complex datasets.

Many organizations are eager to implement machine learning in their business operations. However, effective application requires careful due diligence to ensure a thorough understanding of the data generated by underlying business processes. Ideally, the characteristics of the data should inform the choice of modeling approach. In practice, though, modeling techniques are often predetermined for various reasons, and the focus shifts to adapting the data to fit the chosen model, rather than the other way around.

I started a repository with recipes for descriptive analytics. I hope to add to this incrementally. To begin with I picked a transactions dataset from a national retailer in Brazil. You can check out the narrative and notebook.