The Data-backed Approach to Product Development

Learn the key steps to spot trends, understand consumer drivers, and optimize products for market success.

December 18, 2023

The Data-backed Approach to Product Development

Kaushik Boruah of LatentView sheds light on the high failure rate of new products and the critical role data plays in transforming this landscape. Learn how organizations can navigate challenges and harness data to innovate successfully.

Remember Windows Vista, Crystal Pepsi, Google Glass, and Sony Betamax? All are products from big companies that each had their day in the sun and have since been discontinued after declining popularity. 

Every year, as many as 30,000 new productsOpens a new window hit the market, yet a staggering 85% don’t last in-store for long. In the Consumer Packaged Goods (CPG) sector, taking a product from concept to counter typically takes 18–24 months. Considering the hours of brainstorming, ideation, and capital invested to bring these products to market, that’s a significantly long period. And even after all that, there is still a high chance of failure. Why?

  • Longer experimentation cycle leading to missed opportunities: The process of designing, running, and analyzing experiments hinders progress and delays potential improvements. The risk, in this case, is rarely worth the reward.
  • Siloed decision-making: Each department within the organization needs proper communication and collaboration to make independent decisions.
  • Subjective validation due to focus groups and surveys: The influence of participants’ preferences or opinions leads to subjective rather than objective pre-market evaluations.
  • Data accessibility across enterprises: Scattered data across the organization makes it challenging for teams to access and share information efficiently.
  • Challenges mapping data from internal and external sources: Organizations need help combining and analyzing data effectively from diverse systems and formats.

In a recent LatentView webinarOpens a new window on customer centricity, we discussed organizations’ lack of understanding of their customers from all perspectives. Our experience shows that less than 10% of organizations have a 360-degree view of the customer. Of that 10%, fewer than 5% use consumer data to drive business. 

As a result, the right consumer trend is not anticipated at the right time — reducing customer satisfaction and increasing churn and innovation failure. CPG brands need a holistic framework that weaves data into the innovation process and pulls out insights to address consumer needs as they happen and not after.

See More: Engineering Leaders’ Guide to Cultural Change 

Rolling the Data Dice

CPG organizations often rely on mergers and acquisitions (M&A) to foster innovation, cater to emerging customer segments, expand their product portfolio, consolidate business, and drive growth within the combined entity. Data-driven innovation and analytics play a significant role in this process and are a competitive advantage for those who choose to take advantage of it, enabling organizations to understand customers’ needs and preferences better. While the journey toward innovation isn’t linear, it can be simplified into four essential steps.

1. Trendspotting: First, by tapping into the data derived from social listening through social media channels, specialty e-commerce sites, industry-specific blogs, consumer forums, research papers, and patents, organizations can identify emerging trends before they hit the mainstream.

2. Identifying purchase drivers: Once the trends are identified, brands must consider what drives consumer purchase decisions. Consumer buying behavior is motivated by various factors, sometimes unknown to the brand. Data analysis helps determine the significance of these EFT (emotional, functional, and technical) drivers, enabling organizations to focus on the ones with the most potential for revenue generation and creating a strong brand recall. 

  • Emotional drivers: Whether the product delivers a satisfying, positive experience to the buyer.
  • Functional drivers: Whether the product is reliable and offers value for money. Value can be defined as utility, user-friendliness, or functionality.
  • Technical drivers: Whether the product is innovative and beneficial to the buyer, like vitamins, supplements, etc.

3. Determining opportunity spaces: Next, through white space analysis, organizations can uncover untapped opportunities where a business can innovate, expand, upsell, and position its new products.

4. Establishing product features: Finally, aggregate data allows organizations to identify the characteristics of products, like ingredients, benefits, packaging, and claims (such as organic, vegan, etc.), that are attractive to consumers and have the potential for long-term success.

Connecting the Dots: Analytics and Product Optimization

Analytics empowers organizations to fine-tune their product development within the constraints of manufacturing, costs, and market conditions while aligning their brand with customer expectations. To do so, brands should look at the following:

  • Consumer feedback data: Social media posts, e-commerce reviews, and ratings offer valuable insights into customer mindsets, satisfaction levels, and where peer recommendations are trending.
  • Market performance data: Companies like Nielsen, Circana, and Numerator share invaluable point-of-sale (POS) and household data that help organizations understand market dynamics, product performance, and financial performance. These include data on retail sales, market share, purchases, and promotions and should be built into the brand’s optimization strategy. 
  • Internal insights: By leveraging internal data from an organization’s R&D database, like surveys, focus group responses, supplier information, product shipments, and CRM information, they can get deeper insights into where product development and customer preferences intersect. 

Launching to Success

The last critical step before bringing a new product to market is to validate consumer acceptance. Enter Marketing. Marketing is responsible for overseeing consumer sentiment across the life of the launch — from market research, product positioning, defining the unique selling point, building awareness, and supporting sales with collateral, promotion, and digital marketing to post-launch feedback and monitoring. Effective marketing is critical for defining the target audience, building awareness and demand, and boosting sales.

Purposeful Data for Optimal Impact

Today, organizations generate massive amounts of data from several sources, but the real question is whether the data quality fits its intended purpose. Organizations must identify which sources provide the right data to achieve the desired outcome. Only then can data be a valuable contributor to growth. 

Data sources have always existed. But analytics make sense of this data and drive innovation. Only with the right insights does data become a valuable asset for ideation, research, and decision-making that act as catalysts to identify customer needs and market gaps, opening doors for smart innovation.

How are you leveraging data in your product development journey? Share your insights on FacebookOpens a new window , XOpens a new window , and LinkedInOpens a new window .

Image Source: Shutterstock

MORE ON PRODUCT DEVELOPMENT

Kaushik Boruah
Kaushik Boruah

CPG and Hospitality Practice Lead, LatentView

Kaushik is a CPG and Hospitality Practice Lead at LatentView Analytics as well as a Data Science leader focusing on AI at scale and building digital and analytics solutions across CPG and Hospitality. Experienced in helping leading organizations gain competitive advantage and address pain points through effective analytics interventions and intelligent data mining. He is driven to work in an environment wherein he can leverage his expertise in data analytics to solve challenging business problems, adding value to the organization by contributing to its growth and improving his skills in the process.
Take me to Community
Do you still have questions? Head over to the Spiceworks Community to find answers.