Presentations

Go to day 1 presentationsGo to day 2 presentationsGo to day 3 presentations
Day 1

Wednesday, May 1

8:00 am

Breakfast

8:00 am

Breakfast

9:00 am

Welcome remarks

Bryan Orme
Sawtooth Software

9:00 am

Welcome remarks

Bryan Orme
Sawtooth Software

9:20 am

Review of Quant Applications for LLMs in Market Research

LLMs are impacting nearly every area of market research. There are two key areas where they are impacting the quantitative domain specifically: analysis of dark data (e.g. open ended responses) at scale, and “synthetic responses”. We’ll review different approaches to the application of LLMs to open-ended coding using hands-on examples, and evaluate when each approach is best used. We will then briefly review the existing literature and state of the art on synthetic respondents.

Kevin Karty
Intuify

9:20 am

Review of Quant Applications for LLMs in Market Research

LLMs are impacting nearly every area of market research. There are two key areas where they are impacting the quantitative domain specifically: analysis of dark data (e.g. open ended responses) at scale, and “synthetic responses”. We’ll review different approaches to the application of LLMs to open-ended coding using hands-on examples, and evaluate when each approach is best used. We will then briefly review the existing literature and state of the art on synthetic respondents.

Kevin Karty
Intuify

9:55 am

A “How To” Guide for Catching Cheaters

Data fraud is penetrating every aspect of our industry, including panels, customer databases, and phone research. With no safe place to turn, researchers are left to vet the quality of respondents on our own. The methods cheaters use evolve over time, making it a challenge to keep up with them. This paper summarizes current approaches you can apply to detect data fraud, both real-time and during data analysis.

Deb Ploskonka
Cambia Information Group
Holly Smith
Cambia Information Group
Karlan Witt
Cambia Information Group

9:55 am

A “How To” Guide for Catching Cheaters

Data fraud is penetrating every aspect of our industry, including panels, customer databases, and phone research. With no safe place to turn, researchers are left to vet the quality of respondents on our own. The methods cheaters use evolve over time, making it a challenge to keep up with them. This paper summarizes current approaches you can apply to detect data fraud, both real-time and during data analysis.

Deb Ploskonka
Cambia Information Group
Holly Smith
Cambia Information Group
Karlan Witt
Cambia Information Group

10:30 am

Break

10:30 am

Break

11:00 am

A Qualitative Assessment of Conjoint Surveys: Unlocking Respondent View

When it comes to online surveys, we need to trust that respondents understand and interpret the tasks the way we expect them to. To investigate this, we conduct a series of qualitative interviews aimed to observe how respondents answer, understand and perceive conjoint exercises. Our findings, among others, show that instructions are crucial but often too lengthy, choices depend on anchoring and choices are driven by variety seeking or opportunity to trial new things.

Egle Meskauskaite
SKIM
Remco Don
SKIM

11:00 am

A Qualitative Assessment of Conjoint Surveys: Unlocking Respondent View

When it comes to online surveys, we need to trust that respondents understand and interpret the tasks the way we expect them to. To investigate this, we conduct a series of qualitative interviews aimed to observe how respondents answer, understand and perceive conjoint exercises. Our findings, among others, show that instructions are crucial but often too lengthy, choices depend on anchoring and choices are driven by variety seeking or opportunity to trial new things.

Egle Meskauskaite
SKIM
Remco Don
SKIM

11:30 am

Visibly Better: Improving Conjoint Experiments with Eye Tracking

This research paper explores the potential of utilizing eye-tracking technology to advance conjoint modeling. It explores how eye-tracking data can enhance data quality, validate models, and improve UX in conjoint questionnaires. Key inquiries include participant attention to text and images, the emergence of simplification heuristics, attribute sequencing effects, and the impact of pre-exercise learning sections. The study offers valuable insights into the integration of eye tracking in conjoint research and its implications on model performance.

Neli Dilkova-Gnoyke
Factworks GmbH
Alex Wendland
Factworks GmbH

11:30 am

Visibly Better: Improving Conjoint Experiments with Eye Tracking

This research paper explores the potential of utilizing eye-tracking technology to advance conjoint modeling. It explores how eye-tracking data can enhance data quality, validate models, and improve UX in conjoint questionnaires. Key inquiries include participant attention to text and images, the emergence of simplification heuristics, attribute sequencing effects, and the impact of pre-exercise learning sections. The study offers valuable insights into the integration of eye tracking in conjoint research and its implications on model performance.

Neli Dilkova-Gnoyke
Factworks GmbH
Alex Wendland
Factworks GmbH

12:00 pm

But What If – Using Situational MaxDiff to Understand How Needs Vary across Settings

Product needs vary not only between individuals but can also vary within the same individual based on the situation in which the product is used. We introduce a new method which enables researchers to address this concern: Situational MaxDiff. Situational MaxDiff investigates participant needs based in pre-defined situations. We present a method that leverages pooled data to keep questionnaire time manageable and still produce precise findings which can improve segmentation findings.

Stefan Meissner
GfK SE – An NIQ Company

12:00 pm

But What If – Using Situational MaxDiff to Understand How Needs Vary across Settings

Product needs vary not only between individuals but can also vary within the same individual based on the situation in which the product is used. We introduce a new method which enables researchers to address this concern: Situational MaxDiff. Situational MaxDiff investigates participant needs based in pre-defined situations. We present a method that leverages pooled data to keep questionnaire time manageable and still produce precise findings which can improve segmentation findings.

Stefan Meissner
GfK SE – An NIQ Company

12:30 pm

Lunch

12:30 pm

Lunch

2:00 pm

Surveys for Generation Z, Evaluating Various Mobile-First Designs

We will explore a variety of improvements that can be employed to improve the quality of a mobile questionnaire until it can be described as “mobile first” instead of just “mobile friendly”. We will examine KPIs like dropout rate and data quality both in old studies and in a custom research specifically designed for this purpose. Using this we will quantify the benefits for and identify specific approaches to improve mobile surveys.

Joris van Gool
SKIM

2:00 pm

Surveys for Generation Z, Evaluating Various Mobile-First Designs

We will explore a variety of improvements that can be employed to improve the quality of a mobile questionnaire until it can be described as “mobile first” instead of just “mobile friendly”. We will examine KPIs like dropout rate and data quality both in old studies and in a custom research specifically designed for this purpose. Using this we will quantify the benefits for and identify specific approaches to improve mobile surveys.

Joris van Gool
SKIM

2:30 pm

Artificial Intelligence and Open-Ended Responses in Survey Data Analysis: Topic Analysis and Sentiment Analysis Using AI

The purpose of the presentation will be to present the use and capacity of Artificial Intelligence (AI) on open-ended responses in survey data, particularly as it relates to the analysis stage of a survey data project. Moreover, the presentation will introduce the audience to best practices in the use of two open-ended analysis techniques: topic analysis and sentiment analysis.

Gerardo Martinez Cordeiro
Hanover Research

2:30 pm

Artificial Intelligence and Open-Ended Responses in Survey Data Analysis: Topic Analysis and Sentiment Analysis Using AI

The purpose of the presentation will be to present the use and capacity of Artificial Intelligence (AI) on open-ended responses in survey data, particularly as it relates to the analysis stage of a survey data project. Moreover, the presentation will introduce the audience to best practices in the use of two open-ended analysis techniques: topic analysis and sentiment analysis.

Gerardo Martinez Cordeiro
Hanover Research

3:00 pm

Empowering Market Research with Generative AI: A Paradigm Shift in Consumer Insights

This paper explores the impact of GEN AI in research projects, highlighting real-world applications and insights. It covers questionnaire development, dynamic surveys, data quality management, and text analytics. Key findings include the effectiveness of AI-generated questionnaires, the advantages of dynamic surveys in capturing respondent sentiment, and the potential of fine-tuned models for data quality management. Additionally, machine-driven text analytics shows promise in sentiment analysis and theme identification, achieving near human-level performance on larger datasets.

Mohit Shant
Insights Curry
Mohd. Faisal
Insights Curry

3:00 pm

Empowering Market Research with Generative AI: A Paradigm Shift in Consumer Insights

This paper explores the impact of GEN AI in research projects, highlighting real-world applications and insights. It covers questionnaire development, dynamic surveys, data quality management, and text analytics. Key findings include the effectiveness of AI-generated questionnaires, the advantages of dynamic surveys in capturing respondent sentiment, and the potential of fine-tuned models for data quality management. Additionally, machine-driven text analytics shows promise in sentiment analysis and theme identification, achieving near human-level performance on larger datasets.

Mohit Shant
Insights Curry
Mohd. Faisal
Insights Curry

3:30 pm

Break

3:30 pm

Break

4:00 pm

Revolutionizing Market Research: Immersive Conjoint Driven E-commerce Replicas as a New Frontier

The post-COVID surge in online shopping underscored the need for effective E-Commerce websites, opening research avenues beyond traditional methods. To address this, we developed an immersive Conjoint driven E-commerce replica for surveys. This approach offers valuable behavioural insights and extends a range of research possibilities, from pricing and user engagement metrics. The paper delves into its methodology, initial findings, and scalability, with the potential to transform market research and enhance e-commerce customer insights.

Tarun Khanna
Knowledge Excel
Rashmi Sharma
Knowledge Excel
Saurabh Aggarwal
Knowledge Excel

4:00 pm

Revolutionizing Market Research: Immersive Conjoint Driven E-commerce Replicas as a New Frontier

The post-COVID surge in online shopping underscored the need for effective E-Commerce websites, opening research avenues beyond traditional methods. To address this, we developed an immersive Conjoint driven E-commerce replica for surveys. This approach offers valuable behavioural insights and extends a range of research possibilities, from pricing and user engagement metrics. The paper delves into its methodology, initial findings, and scalability, with the potential to transform market research and enhance e-commerce customer insights.

Tarun Khanna
Knowledge Excel
Rashmi Sharma
Knowledge Excel
Saurabh Aggarwal
Knowledge Excel

4:30 pm

Share of Search: The New Crown Jewel or the Emperor’s New Clothes?

Share of Search (SOS) is currently generating significant buzz in the marketing industry. Marketing effectiveness expert, Les Binet, has suggested that Excess Share of Search (ESOS) can predict future changes in market share. We test Binet's theory using hundreds of technology brands and show how we can enhance the predictive accuracy further by improving the ESOS methodology. Our approach offers a highly valuable quick and cost-effective alternative to costly and time-consuming bespoke forecasting models.

James Pitcher
GfK
Alexandra Chirilov
GfK

4:30 pm

Share of Search: The New Crown Jewel or the Emperor’s New Clothes?

Share of Search (SOS) is currently generating significant buzz in the marketing industry. Marketing effectiveness expert, Les Binet, has suggested that Excess Share of Search (ESOS) can predict future changes in market share. We test Binet's theory using hundreds of technology brands and show how we can enhance the predictive accuracy further by improving the ESOS methodology. Our approach offers a highly valuable quick and cost-effective alternative to costly and time-consuming bespoke forecasting models.

James Pitcher
GfK
Alexandra Chirilov
GfK

5:00 pm

Identifying Winning Feature Combinations with Combined Reach of Item Sets

This research introduces a practical method for assessing the collective impact of feature sets on adoption rates, extending beyond traditional MaxDiff and TURF analysis. While TURF assumes that one feature can reach a customer, our approach recognizes the necessity of multiple must-have features for convincing consumers. Particularly valuable when evaluating extensive feature lists with numerous "must-haves" and when a Conjoint exercise isn't feasible, our method surpasses conventional techniques such as simple select questions or TURF.

Alex Wendland
Factworks GmbH
Neli Dilkova-Gnoyke
Factworks GmbH

5:00 pm

Identifying Winning Feature Combinations with Combined Reach of Item Sets

This research introduces a practical method for assessing the collective impact of feature sets on adoption rates, extending beyond traditional MaxDiff and TURF analysis. While TURF assumes that one feature can reach a customer, our approach recognizes the necessity of multiple must-have features for convincing consumers. Particularly valuable when evaluating extensive feature lists with numerous "must-haves" and when a Conjoint exercise isn't feasible, our method surpasses conventional techniques such as simple select questions or TURF.

Alex Wendland
Factworks GmbH
Neli Dilkova-Gnoyke
Factworks GmbH

5:30 pm

Machine Learning in Market Segmentation Research

This presentation shows how to improve market segmentation studies with the judicious use of machine learning methods. We start by outlining the four steps of a typical market segmentation process: • Selecting basis variables • Creating segments • Profiling segments • Predicting segment membership Traditionally, the industry used taxonomic and statistical methods for these steps. In a few specific instances, those methods are the correct ones to use. But they don’t generalize well to all or even many cases of market segmentation. Frequently over-used in lazy (segment creation via cluster analysis) or simply incorrect (variable selection via factor analysis) ways, they almost always be improved upon. Machine learning methods can improve the segmentation research process at various of its four steps: • Sometimes as additional tools to enhance what we were able to do before with traditional statistics (e.g., decision trees for segment creation) • Sometimes as improvements over what traditional statistics can do (e.g., random forests for segment prediction) • Sometimes to do things properly that misunderstood statistical methods are inappropriately tasked to do (e.g., factor analysis is frequently, and incorrectly, thought to be a data pre-treatment method for segmentation analysis, but machine learning techniques are the vastly better way to select basis variables).

Keith Chrzan
Sawtooth Software

5:30 pm

Machine Learning in Market Segmentation Research

This presentation shows how to improve market segmentation studies with the judicious use of machine learning methods. We start by outlining the four steps of a typical market segmentation process: • Selecting basis variables • Creating segments • Profiling segments • Predicting segment membership Traditionally, the industry used taxonomic and statistical methods for these steps. In a few specific instances, those methods are the correct ones to use. But they don’t generalize well to all or even many cases of market segmentation. Frequently over-used in lazy (segment creation via cluster analysis) or simply incorrect (variable selection via factor analysis) ways, they almost always be improved upon. Machine learning methods can improve the segmentation research process at various of its four steps: • Sometimes as additional tools to enhance what we were able to do before with traditional statistics (e.g., decision trees for segment creation) • Sometimes as improvements over what traditional statistics can do (e.g., random forests for segment prediction) • Sometimes to do things properly that misunderstood statistical methods are inappropriately tasked to do (e.g., factor analysis is frequently, and incorrectly, thought to be a data pre-treatment method for segmentation analysis, but machine learning techniques are the vastly better way to select basis variables).

Keith Chrzan
Sawtooth Software

5:30 pm

Key Drivers Analysis in LHS

Drivers analysis is a key part of any analyst’s toolbox, but many of the techniques in use today can still be demonstrably improved upon. In order to assist with this, Key Drivers Analysis (incorporating Johnson’s e) is now available within Lighthouse Studio v9.16. Join Walter Williams and Dean Tindall as they introduce you to this newly released module and the reasons why we chose to implement this particular method.

Dean Tindall
Sawtooth Software
Walter Williams
Sawtooth Software

5:30 pm

Key Drivers Analysis in LHS

Drivers analysis is a key part of any analyst’s toolbox, but many of the techniques in use today can still be demonstrably improved upon. In order to assist with this, Key Drivers Analysis (incorporating Johnson’s e) is now available within Lighthouse Studio v9.16. Join Walter Williams and Dean Tindall as they introduce you to this newly released module and the reasons why we chose to implement this particular method.

Dean Tindall
Sawtooth Software
Walter Williams
Sawtooth Software

7:00 pm

Evening Reception at Casa Rio

7:00 pm

Evening Reception at Casa Rio

Back to Top
Day 2

Thursday, May 2

8:00 am

Breakfast

8:00 am

Breakfast

9:00 am

Judgement Day: The Machines Have Arrived – But How Good Are They At Answering Choice Experiments

Choice modelling is often used to optimise the configuration of products/services or to create a hierarchy ranking of items. This paper will investigate the use of large language models (LLMs) to replicate the results of previous Conjoint and MaxDiff projects. We will test whether LLMs can be used to generate realistic responses to tasks, whether responses are consistent across LLMs and parameter settings, and whether you can train LLM’s to produce better quality data.

Chris Moore
Ipsos UK
Cameron Stronge
Ipsos UK
Manjula Bhudiya
Ipsos UK

9:00 am

Judgement Day: The Machines Have Arrived – But How Good Are They At Answering Choice Experiments

Choice modelling is often used to optimise the configuration of products/services or to create a hierarchy ranking of items. This paper will investigate the use of large language models (LLMs) to replicate the results of previous Conjoint and MaxDiff projects. We will test whether LLMs can be used to generate realistic responses to tasks, whether responses are consistent across LLMs and parameter settings, and whether you can train LLM’s to produce better quality data.

Chris Moore
Ipsos UK
Cameron Stronge
Ipsos UK
Manjula Bhudiya
Ipsos UK

9:45 am

Comparing AI-Generated Results to Survey Research in CPG Product Pricing

We examine how AI large language models compare with direct stated pricing models such as Van Westendorp and indirect derived pricing models such as a price brand trade-off exercise when determining optimal pricing recommendations for a product line across a variety of distribution channels.

Kathryn Kaul-Goodman
Harris Poll
Jacob Nelson
Harris Poll
Edward Paul Johnson
Harris Poll

9:45 am

Comparing AI-Generated Results to Survey Research in CPG Product Pricing

We examine how AI large language models compare with direct stated pricing models such as Van Westendorp and indirect derived pricing models such as a price brand trade-off exercise when determining optimal pricing recommendations for a product line across a variety of distribution channels.

Kathryn Kaul-Goodman
Harris Poll
Jacob Nelson
Harris Poll
Edward Paul Johnson
Harris Poll

10:30 am

Break

10:30 am

Break

11:00 am

Complete Level Overlap with Color Coding: Validation, Extension and a New Super Power

Designs where many attributes in a task overlap completely and where overlapped levels are shaded reportedly have several benefits: the less complex choice task results in reduced response error, dropouts and attribute non-attendance. We seek to confirm these findings and extend them to see whether these designs will be immune to the ill-effects of dominating attributes and whether they will have fewer respondents giving random responses.

Keith Chrzan
Sawtooth Software
Dan Yardley
Sawtooth Software

11:00 am

Complete Level Overlap with Color Coding: Validation, Extension and a New Super Power

Designs where many attributes in a task overlap completely and where overlapped levels are shaded reportedly have several benefits: the less complex choice task results in reduced response error, dropouts and attribute non-attendance. We seek to confirm these findings and extend them to see whether these designs will be immune to the ill-effects of dominating attributes and whether they will have fewer respondents giving random responses.

Keith Chrzan
Sawtooth Software
Dan Yardley
Sawtooth Software

11:45 am

Using Seeded Items to Improve Express Best Worst Designs

Previous research on many-item Best Worst tasks has shown that Sparse BW designs have generally outperformed Express BW designs, especially regarding out-of-sample predictions. Recently, a suggestion was made to improve Express BW designs by including a small, fixed number of items [3-5] across all respondents. We undertake this research to ascertain whether item seeding results in better out-of-sample predictions than traditional Express BW designs have without this seeding.

Jon Godin
Numerious
Megan Peitz
Numerious
Thomas Eagle
Eagle Analytics of California

11:45 am

Using Seeded Items to Improve Express Best Worst Designs

Previous research on many-item Best Worst tasks has shown that Sparse BW designs have generally outperformed Express BW designs, especially regarding out-of-sample predictions. Recently, a suggestion was made to improve Express BW designs by including a small, fixed number of items [3-5] across all respondents. We undertake this research to ascertain whether item seeding results in better out-of-sample predictions than traditional Express BW designs have without this seeding.

Jon Godin
Numerious
Megan Peitz
Numerious
Thomas Eagle
Eagle Analytics of California

12:30 pm

Lunch

12:30 pm

Lunch

2:00 pm

Comparing Pricing Approaches in Conjoint Analysis

Building on our paper presented at the A&I Conference 2023, this research extends the application of proportional pricing in conjoint analysis to manufacturer brands. It examines the effects of omitting explicit monetary values on consumer choice behaviour and, consequently, price elasticity. It also assesses how well proportional pricing method predicts real-world market performance, validating it as a useful alternative especially for complex product offerings where traditional monetary pricing is elusive.

Alexandra Chirilov
GfK
James Pitcher
GfK

2:00 pm

Comparing Pricing Approaches in Conjoint Analysis

Building on our paper presented at the A&I Conference 2023, this research extends the application of proportional pricing in conjoint analysis to manufacturer brands. It examines the effects of omitting explicit monetary values on consumer choice behaviour and, consequently, price elasticity. It also assesses how well proportional pricing method predicts real-world market performance, validating it as a useful alternative especially for complex product offerings where traditional monetary pricing is elusive.

Alexandra Chirilov
GfK
James Pitcher
GfK

2:45 pm

Yoshimi* Battles the Survey Bots: How you can work to defeat those evil-natured robots in your online survey samples

Advances in automation technology and artificial intelligence have made it easier to create and deploy bots for various purposes, including survey participation. As AI technology becomes more sophisticated, survey bots can become more intelligent and difficult to detect. This can exacerbate the challenges associated with identifying and preventing bots from participating in surveys, which is a concern for the market research industry. Consequently, market researchers need to continuously adapt and develop more robust methods to distinguish between genuine survey participants and automated bots. However, AI bots also have weaknesses (so far) and we at Numerious developed a new approach that is designed to exploit their weaknesses.

Leyla Eden
Numerious
Daniel Barkley
Numerious
Trevor Olsen
Numerious

2:45 pm

Yoshimi* Battles the Survey Bots: How you can work to defeat those evil-natured robots in your online survey samples

Advances in automation technology and artificial intelligence have made it easier to create and deploy bots for various purposes, including survey participation. As AI technology becomes more sophisticated, survey bots can become more intelligent and difficult to detect. This can exacerbate the challenges associated with identifying and preventing bots from participating in surveys, which is a concern for the market research industry. Consequently, market researchers need to continuously adapt and develop more robust methods to distinguish between genuine survey participants and automated bots. However, AI bots also have weaknesses (so far) and we at Numerious developed a new approach that is designed to exploit their weaknesses.

Leyla Eden
Numerious
Daniel Barkley
Numerious
Trevor Olsen
Numerious

3:30 pm

Break

3:30 pm

Break

4:00 pm

Fairness in Clustering – Opportunities for Application in Market Segmentation

Fairness in clustering refers to the requirement that certain respondent types like minority grouping have adequate representation across clusters to avoid bias. It has emerged as an actively researched area in last few years. Fairness can be a practical concern in market segmentation. I will briefly introduce fair clustering, then focus on comparing selected promising and practical algorithms on both benchmark and real data sets. I will discuss method choices, fairness specification, computation, and implementation.

Ming Shan
Hall & Partners

4:00 pm

Fairness in Clustering – Opportunities for Application in Market Segmentation

Fairness in clustering refers to the requirement that certain respondent types like minority grouping have adequate representation across clusters to avoid bias. It has emerged as an actively researched area in last few years. Fairness can be a practical concern in market segmentation. I will briefly introduce fair clustering, then focus on comparing selected promising and practical algorithms on both benchmark and real data sets. I will discuss method choices, fairness specification, computation, and implementation.

Ming Shan
Hall & Partners

4:30 pm

Navigating the Social Media Data Landscape: A Quantitative Approach to Insight Generation

Social media platforms have emerged as influential spaces where individuals and brands interact, share information, and influence opinions. The huge amount of data generated on these platforms contains rich information but presents challenges in harnessing it. We introduced a quantitative framework to navigate this complexity, highlighting the process of data collection, analysis, and insight generation, culminating in the development of a brand benchmarking framework. The framework can be applied to any industry with relevant customizations.

Rachin Gupta
StatWorld Analytics, LLC.
Rajat Goel
StatWorld Analytics, LLC.

4:30 pm

Navigating the Social Media Data Landscape: A Quantitative Approach to Insight Generation

Social media platforms have emerged as influential spaces where individuals and brands interact, share information, and influence opinions. The huge amount of data generated on these platforms contains rich information but presents challenges in harnessing it. We introduced a quantitative framework to navigate this complexity, highlighting the process of data collection, analysis, and insight generation, culminating in the development of a brand benchmarking framework. The framework can be applied to any industry with relevant customizations.

Rachin Gupta
StatWorld Analytics, LLC.
Rajat Goel
StatWorld Analytics, LLC.

5:00 pm

Price-Group Estimation Approach for Price Attribute in Choice Models Using Alternative Specific Design (ASD)

To capture consumer’s response to psychological price points, ideally, every Brand-SKU X price point should be sufficiently exposed and estimated as part-worth (non-linear) for every respondent. Many a times, this is not feasible. So, we explored creating ASD price-groups such that brand-SKUs with similar tested price-levels and similar response to these price changes(after studying count analysis), fall in same price-group. We then estimate utilities for price-groups. This method restructures the same information that is already captured, without increasing sample size and performs well at estimating psychological price points.

Surbhi Minocha
KANTAR

5:00 pm

Price-Group Estimation Approach for Price Attribute in Choice Models Using Alternative Specific Design (ASD)

To capture consumer’s response to psychological price points, ideally, every Brand-SKU X price point should be sufficiently exposed and estimated as part-worth (non-linear) for every respondent. Many a times, this is not feasible. So, we explored creating ASD price-groups such that brand-SKUs with similar tested price-levels and similar response to these price changes(after studying count analysis), fall in same price-group. We then estimate utilities for price-groups. This method restructures the same information that is already captured, without increasing sample size and performs well at estimating psychological price points.

Surbhi Minocha
KANTAR

5:30 pm

Introduction to MBC (Menu-Based Choice)

As you gain more experience in choice modeling, you come to realize that CBC (Choice-Based Conjoint) is a specific and limited case within a broader family of choice experiments. Sometimes clients push examples upon you that bring you to this realization. Sometimes you see examples at conferences and in white papers that open your eyes and spur your interest. There is, of course, the classic example resembling a fast-food restaurant where respondents can pick one of multiple items to add to their cart for purchase. There are other examples that don’t immediately strike you as being an MBC, but that exceed CBC’s capabilities and can be handled with Sawtooth Software’s MBC software. Examples include asking respondents if they will buy the product, rent the product, or do neither. Or, situational choice in which the choice of multiple fixed alternatives is a function of respondent characteristics, buying situation characteristics, or patient characteristics. Join Bryan Orme in this 45-minute clinic to learn about the possibilities with MBC!

Bryan Orme
Sawtooth Software

5:30 pm

Introduction to MBC (Menu-Based Choice)

As you gain more experience in choice modeling, you come to realize that CBC (Choice-Based Conjoint) is a specific and limited case within a broader family of choice experiments. Sometimes clients push examples upon you that bring you to this realization. Sometimes you see examples at conferences and in white papers that open your eyes and spur your interest. There is, of course, the classic example resembling a fast-food restaurant where respondents can pick one of multiple items to add to their cart for purchase. There are other examples that don’t immediately strike you as being an MBC, but that exceed CBC’s capabilities and can be handled with Sawtooth Software’s MBC software. Examples include asking respondents if they will buy the product, rent the product, or do neither. Or, situational choice in which the choice of multiple fixed alternatives is a function of respondent characteristics, buying situation characteristics, or patient characteristics. Join Bryan Orme in this 45-minute clinic to learn about the possibilities with MBC!

Bryan Orme
Sawtooth Software

7:00 pm

Evening Reception at the Alamo

7:00 pm

Evening Reception at the Alamo

Back to Top
Day 3

Friday, May 3

8:00 am

Breakfast

8:00 am

Breakfast

9:00 am

Holistic Conjoint

In complex choice situations it is unlikely that consumers will break down each attribute and each level, assign values to the levels, integrate these values and then choose the highest overall value. Instead, respondents may, at least partially, make choices based on a holistic evaluation of the features. We found that the holistic variable was highly statistically significant, and we found several tipping points in the utility function of the holistic variable.

Marco Vriens
Kwantum
Felix Eggers
Copenhagen Business School
Darin Mills
Illuminas

9:00 am

Holistic Conjoint

In complex choice situations it is unlikely that consumers will break down each attribute and each level, assign values to the levels, integrate these values and then choose the highest overall value. Instead, respondents may, at least partially, make choices based on a holistic evaluation of the features. We found that the holistic variable was highly statistically significant, and we found several tipping points in the utility function of the holistic variable.

Marco Vriens
Kwantum
Felix Eggers
Copenhagen Business School
Darin Mills
Illuminas

9:30 am

Extracting Meaningful Segments from HB Utilities

In this presentation we illustrate how latent class (LC) clustering should be performed on HB utilities to avoid strange results, and describe the situations where Latent GOLD’s scale-adjusted (SALC) model would be expected to result in more meaningful segments. Our MaxDiff example yields 88% agreement with the gold standard, vs. 60% when clustering is done incorrectly. Attendees will take away new insights into the segmentation process when latent class models are applied to HB utilities.

Jay Magidson
Statistical Innovations
Jeroen K. Vermunt
Tilburg University

9:30 am

Extracting Meaningful Segments from HB Utilities

In this presentation we illustrate how latent class (LC) clustering should be performed on HB utilities to avoid strange results, and describe the situations where Latent GOLD’s scale-adjusted (SALC) model would be expected to result in more meaningful segments. Our MaxDiff example yields 88% agreement with the gold standard, vs. 60% when clustering is done incorrectly. Attendees will take away new insights into the segmentation process when latent class models are applied to HB utilities.

Jay Magidson
Statistical Innovations
Jeroen K. Vermunt
Tilburg University

10:00 am

Clearing out the Garbage Can: An empirical test of variable selection and data reduction in market segmentation

As conference attendees have seen over the past few years, these are exciting times for practitioners that look to improve our toolkit for marketing segmentation. But are we including the right variables in our segmentation studies? This paper will look at combining automated search algorithms and variations on latent class analysis to see whether we can demonstratively present better resolution in our findings compared to previous segmentation work with commercially accessible software.

Stuart Drucker
Advalytics

10:00 am

Clearing out the Garbage Can: An empirical test of variable selection and data reduction in market segmentation

As conference attendees have seen over the past few years, these are exciting times for practitioners that look to improve our toolkit for marketing segmentation. But are we including the right variables in our segmentation studies? This paper will look at combining automated search algorithms and variations on latent class analysis to see whether we can demonstratively present better resolution in our findings compared to previous segmentation work with commercially accessible software.

Stuart Drucker
Advalytics

10:30 am

Break

10:30 am

Break

11:00 am

Respondent Fatigue in Choice-Based Conjoint – When and how does it affect the results?

Respondents are generally assumed to become tired by taking repeating CBC exercises. However, prior research points to the absence of Respondent Fatigue in the CBC part of surveys. This research confronts this by stress-testing respondents in CBC through 3 studies with 32 tasks each and different levels of complexity. We tested for Stated Fatigue, Implied Fatigue (trap questions), and Derived Fatigue (model fit). Results show Respondent Fatigue exists, but the implications are limited for practitioners.

Carl Johan Ekstromer
SKIM

11:00 am

Respondent Fatigue in Choice-Based Conjoint – When and how does it affect the results?

Respondents are generally assumed to become tired by taking repeating CBC exercises. However, prior research points to the absence of Respondent Fatigue in the CBC part of surveys. This research confronts this by stress-testing respondents in CBC through 3 studies with 32 tasks each and different levels of complexity. We tested for Stated Fatigue, Implied Fatigue (trap questions), and Derived Fatigue (model fit). Results show Respondent Fatigue exists, but the implications are limited for practitioners.

Carl Johan Ekstromer
SKIM

11:30 am

60 Years of Conjoint: Where We Came from and Where We Are

Six decades ago, Luce and Tuckey's seminal paper marked the inception of conjoint analysis. Now, it's time to delve into the historical evolution of conjoint analysis over the past 60 years. In particular, we'll explore a concise history of advancements in various conjoint models, the estimation of part-worth utilities, the creation of experimental designs, and how these innovations have shaped research outcomes. We'll also examine which of the challenges that emerged during this period have been successfully addressed and which questions still remain open.

Peter Kurz
bms marketing research + strategy

11:30 am

60 Years of Conjoint: Where We Came from and Where We Are

Six decades ago, Luce and Tuckey's seminal paper marked the inception of conjoint analysis. Now, it's time to delve into the historical evolution of conjoint analysis over the past 60 years. In particular, we'll explore a concise history of advancements in various conjoint models, the estimation of part-worth utilities, the creation of experimental designs, and how these innovations have shaped research outcomes. We'll also examine which of the challenges that emerged during this period have been successfully addressed and which questions still remain open.

Peter Kurz
bms marketing research + strategy

12:00 pm

Tribute to Rich Johnson, Best Paper Presentation, and Closing Remarks

Bryan Orme
Sawtooth Software

12:00 pm

Tribute to Rich Johnson, Best Paper Presentation, and Closing Remarks

Bryan Orme
Sawtooth Software
Back to Top
Don’t delay!

Get tickets to attend the 2024 A&I Summit