What to know about AI in retail analytics

With structured data as the foundation, AI enables smarter, faster decision-making across the enterprise. Here’s your guide to future-proof implementation.

Retail data is a necessary foundation for CPG operations and decision-making, providing suppliers with crucial visibility into real-time retail and inventory movements across thousands of touchpoints. It’s vital in an age where shopper behavior and product trends are in constant flux. 

Beyond day-to-day functions, however, the amount of retail data suppliers incur across the omnichannel landscape can be so expansive and granular that it can become overwhelming to analyze and extract meaningful insights from. 

That could be why AI — known for its efficiency and productivity possibilities — presents such a compelling opportunity for the retail and consumer goods industry, with an estimated potential impact of $400 billion to $660 billion annually; driven by AI-supported strategic growth and cost savings through automation of key functions. 

This value hasn’t gone unnoticed. Retailers’ investments in AI to realize this value have led to two key developments: consumer technology that personalizes shopping experiences and drives purchase behavior; and new, more robust data infrastructure and reporting solutions to drive supplier success.

While the wealth of new data made available by retailers is beneficial, their AI-driven visions also raise the bar for performance, demanding faster response times and fulfillment.

AI-powered analytics can accelerate suppliers’ time-to-insight and help them make smarter, faster decisions around demand forecasting, inventory management, and business planning, ensuring the adaptability of their operations. 

But there’s a learning curve — with over 85% of generative AI projects initiated by companies failing to reach production this year.

As we’re beginning to understand, the major roadblock is underdeveloped data that fails to meet the intricate, highly granular requirements of large language models (LLMs) and other similar foundation models. While AI promises extreme automation and industry transformation, thoughtfully structured data is essential for the accurate and integrated use of AI tools in retail analytics.

In this article, we uncover how to harness retail data for fast, reliable, and powerful AI-powered insights and offer clear paths that organizations can take to begin realizing enterprise-wide value today.

The major roadblock [in generative AI project project production] is underdeveloped data that fails to meet the intricate, highly granular requirements of large language models (LLMs) and other similar foundation models.

Data as the foundation

For CPG suppliers, retail data sources span retailers and distributors of all types, each with their own disparate nomenclature and structure. Millions of records of highly granular data that tracks inventory and sales from distribution centers through to individual consumer purchases at the store level are generated every day. Crisp’s data platform ingests, normalizes, and harmonizes this data from multiple sources, creating a unified foundation for analytics and decision-making.

However, suppliers need to take data preparedness a step further to leverage AI’s power in retail analytics. This is where a semantic layer comes in, building consistent, natural business language around disparate data sources, and enriching it with logic and context tailored to the nuances of a unique industry. 

The semantic layer adds crucial meaning and context to normalized data. This is because while a generative AI model might recognize terms like ‘Dollar Velocity’ or ‘Share of Category YoY Growth,’ it needs to have the understanding of their precise definitions and business context to contribute actionable insights. The semantic layer explicitly defines these metrics, establishes their relationships, and encodes the underlying business logic, empowering CPG teams to draw insights that seamlessly inform inventory management, merchandising, growth strategies, and more.

Recognizing this need, Crisp has created the first semantic layer specifically for the retail industry. This added embedded technology provides the context necessary to transform collective retail data sources into natural language queryable information. It can be utilized across departments for fast, reliable insights regardless of technical expertise, heralding a major unlock in AI-driven efficiency.

While a generative AI model might recognize terms like ‘Dollar Velocity’ or ‘Share of Category YoY Growth,’ it needs to have the understanding of their precise definitions and business context to contribute actionable insights. The semantic layer explicitly defines these metrics, establishes their relationships, and encodes necessary underlying business logic.

Interacting with intelligence

With a robust data foundation in place, what comes next? How can businesses leverage AI-ready retail data for even faster, more thoughtful decision-making?

Perhaps you’re experienced in leveraging the best-in-class models from OpenAI, Anthropic, Google, or Microsoft for a range of tasks in your day-to-day operations such as research, ideation, task automation, or code optimization. Maybe you’ve even fine-tuned your own models, using data sets and rules that are proprietary to your organization.

But leveraging generative AI to extract accurate, meaningful information from billions of rows of both historical and real-time data is an entirely different challenge. One that today’s leading data companies are well-positioned to meet head-on.

Microsoft Copilot for Real-Time Intelligence, Snowflake Cortex, and Databricks AI/BI Genie are emerging enterprise solutions that enable the ability to query a constellation of diverse data using plain, high-level business language. This opens possibilities for CPGs to gain a comprehensive understanding of the diverse factors impacting their supply chain and sales performance. Imagine not just asking questions like, “What was the impact of our latest promotion on inventory across distribution centers?” figuratively, in querying data, but literally, leveraging your choice of LLM. These AI-powered tools by top data companies can quickly process vast amounts of data to provide these nuanced answers, uncovering patterns and insights that could otherwise take days of modeling and analysis to uncover. 

Imagine not just asking questions like, “What was the impact of our latest promotion on inventory across distribution centers?” figuratively, in querying data, but literally, leveraging your choice of LLM.

Deep and meaningful possibilities

Where non-technical teams will enjoy interacting with language models for information to fuel daily decision-making, technical audiences like data science engineers and analysts will appreciate a reliable stream of structured intelligence to automate reporting tasks and fuel deeper learning and problem-solving.

Time (freed from ingesting and organizing data, maintaining pipelines, or building semantic modeling) and resources (real-time normalized, harmonized and prepared data across sources) create the ideal conditions to develop rich, predictive analytics for demand forecasting, inventory optimization, and promotional planning.

And since semantic layers offer a common language, there’s opportunity for even greater collaboration within the data engineering community.

The continued rise of Jupyter notebooks for data exploration and model development allows data scientists to read, understand, and run code in a step-by-step manner. It also features visualization capabilities to AI-generate graphs and charts, or offer explanatory summaries.

Open-source retail data models available in Jupyter notebooks are an exciting forefront for cross-industry collaboration. Current developments at Crisp span store clustering, weather analytics, and anomaly detection projects to help customers efficiently access, analyze, and incorporate these models, reducing time and effort to integrate Crisp into their existing workflows and opening the possibilities of future new applications.

Where non-technical teams will enjoy interacting with language models for information to fuel daily decision-making, technical audiences like data science engineers and analysts will appreciate a reliable stream of structured intelligence to automate reporting tasks and fuel deeper learning and problem-solving.

Path to zero waste

How is AI-enabled analytics expected to address critical challenges in the retail industry? Let’s consider the pressing issue of food waste.

By integrating meaningful company information like EDI purchase order (PO) records and retail data, along with external data sets such as syndicated market reports and weather data, AI-enabled analytics can provide a comprehensive view of the supply chain.

Continuous supply chain improvements facilitated by AI could hold the answer to solving food waste — a problem that currently represents 30-40% of the food supply in the United States, and accounts for an estimated 8-10% of greenhouse gas emissions (GHG).

Ready to transform your retail analytics with AI? The first step starts with Crisp — book a demo today.