Agenda
Many real-world purchases aren’t a single pick from a set of alternatives. Customers build bundles, add options and choose à la carte. This hands-on, one-day workshop introduces Menu-Based Choice (MBC), the proven approach to modelling these complex decisions.
You’ll learn when MBC is the right tool to use, how to design menu-based studies, and how to estimate and interpret models using our MBC software. MBC software also can be used for analyzing certain kinds of situational choice experiments, sometimes referred to as polytonomous logit. Through practical exercises, we’ll walk you through building models to test pricing, bundling and product configuration scenarios.
Ideal for market researchers already familiar with conjoint analysis who want to expand their choice-modelling toolkit beyond standard CBC.
This version adds a full suite of segmentation methods. This session starts with the foundations of successful segmentation: measuring and scaling variables well, and choosing which ones to include, drawing on new R&D into variable selection. We’ll then compare the major algorithms, including how each performs under difficult data conditions, and close with live demos of k-means, hierarchical, ensemble, and model-based (latent class) clustering in Lighthouse Studio.
Learn to design, field, and analyze MaxDiff studies for measuring importance and preference, whether with hundreds of respondents or a handful in IDIs and focus groups. We’ll cover when to choose MaxDiff over simpler approaches, segmentation with Discover’s segment finder, and TURF for building item sets that reach the most people. Exercises use the free version of Discover, so there’s nothing to buy. Afterward, take the certification test to earn a Sawtooth MaxDiff certification.
Purpose-built AI respondents can defeat most quality checks, yet a cross-platform study found almost none outside MTurk. Nonetheless, the field lacks scalable detection metrics. This session introduces a first-of-its-kind framework for evaluating detection methods at scale. Scripted bots, commercial agents, and optimised custom builds complete the same instrument as verified humans, and each proposed check is scored by its catch rate while flagging almost no real respondents. The results show which checks to deploy, retire, and layer.
We have all seen them when reviewing survey data: respondents where something does not look right, but there is not enough evidence to remove them. I will test whether deeper patterns within a respondent's answers and similarities between respondents can uncover low-quality responses that standard checks miss. Across multiple studies, I will examine which checks add useful evidence, approaches to calibrating respondent similarity, and whether removing flagged respondents materially changes the research results.
In 2025, we proposed an AI-driven anchored MaxDiff tracker that could evolve its own item list. We ran four waves of a survey with two MaxDiff exercises to track what respondents were happy about and what worried them, with generative AI analyzing open-ends and recommending item swaps each wave. We'll share what worked, how the LLM-analyzed results compared to human-coded results, and our overall thoughts on using genAI to support tracking respondent sentiment with MaxDiff.
Research only creates value if people can understand, remember, and use it. Using segmentation as a controlled test case, we’ll compare traditional profiles with newer activation approaches-including chatbots, video montages, and interactive avatars-on comprehension, application, engagement, confidence, and one-week retention. We’ll also test whether more lifelike or interactive formats encourage unsupported assumptions. The findings will show which approaches genuinely improve activation, which work best for different goals, and which may simply make research more engaging.
How can we get marketing to act on insights offered by the consumer insights team? Stakeholder alignment, trust, credibility and articulating the likely impact may all be necessary but not sufficient. A review of the literature across psychology and organizational science reveals other factors such as how easily insights can be understood, timeliness, overcoming the decision-maker’s own mental model(s), and increasing psychological receptiveness. Combined with 25+ years of client-side and consulting experience, attendees leave with a practical framework for turning accepted findings into actual decisions.
Conjoint studies with many binary features often overpredict preference for feature-rich products and underpredict preference for feature-poor products. We investigate practical solutions to this persistent challenge, combining experimental design adjustments, summed pricing, and attribute non-attendance adjustments. Using both synthetic and real respondents, we explore promising approaches to improve predictive accuracy for a study involving brand, price, and 30 binary attributes. The goal: a straightforward, practitioner-friendly methodology for CBC studies with many binary features and price.
Studies with large product assortments cannot present every alternative without overburdening respondents. Conventional methods reduce the set through preliminary screening, but identifying relevant alternatives generally still requires a prohibitive number of screening tasks. We propose replacing these tasks with an LLM-led conversation that learns each respondent’s preferences from their needs, habits, and desired benefits. The LLM then uses this profile to screen the available items and create a personalized set for the subsequent conjoint exercise.
Digital-twin accuracy depends on both how accuracy is measured and what question the twin is asked. Using multiple survey datasets, the research shows why strong aggregate results can hide weak individual-level accuracy. It then introduces an R²-based answerability screen for evaluating new questions when human answers are unavailable. The screen helps researchers identify which synthetic results are more defensible and which questions still require human research.
Brand maps and drivers analyses are market research staples, but their outputs can be difficult to interpret and act upon. We introduce Moon Plots, a more intuitive alternative to conventional correspondence maps, and share code to create them. We also take Bayesian Networks analysis beyond standard outcome-focused drivers analysis by quantifying what drives intermediate perceptions as well as final outcomes. We show how these approaches create clearer positioning stories and more actionable growth strategies.
The synthetic data conversation has shifted from “does it work?” to “when should I use it?” Answering that takes governance, transparency, and collaboration. A year after introducing the ECHO Index, we share lessons from rigorous evaluation: how iteration improved a provider’s model, how far sample boosting can be pushed in practice, and the challenges with governing digital twins. Attendees will leave equipped with strategies and tools to tackle their next synthetic data study.
Track 1: Morning presenters
Track 2: Afternoon presenters
Track 3: Keith Chrzan, Greg Allenby, and Megan Peitz
In recent versions of Lighthouse Studio we have added the ability to under take a range of Key Driver and Clustering analyses (Including Latent Class!) within the Analysis Manager. However, we plan on expanding this toolset even further, join us an introduction to the new techniques we are adding to the software (Regression, Discriminant Analysis, Factor Analysis, Gradient Boosted Trees, Unsupervised Random Forests), and a demo of how these tools work within Lighthouse Studio.
Join us for an overview of Discover's updates since last conference! We'll cover conjoint and MaxDiff updates, survey invitations, and show you how to connect your AI tool of choice to automate survey programming and recreate surveys you've programmed in other platforms.
We'll also demonstrate various ways you can customize the look, feel and functionality of Discover surveys.
Dual-response conjoint asks a binary follow-up question: "would you actually buy it?" We provide the behavioral model and likelihood for its natural extension to an ordinal follow-up: "how likely are you to buy it?" We compare our approach to two common shortcuts (collapsing the scale to yes/no via top-box, and assigning each scale point a fixed purchase probability) and show that both can bias demand and profit expectations.
Pricing research teams are being asked to do more with less—and nobody wants to wait three months for an answer. Meanwhile, AI monetization decisions are becoming more complex. Through two real-world B2B case studies, we show how we delivered rigorous pricing insights on accelerated timelines, using radically different approaches: a customized menu-based conjoint and rapid qual-plus-quant experimentation. Attendees will learn how to match methodological rigor to decision complexity without sacrificing the confidence business leaders need to act.
We develop a flexible conjoint model that captures how stand-alone attribute values combine to determine product utility. The model accommodates respondent-specific subadditive, additive, and superadditive aggregation. An application to television preferences reveals distinct aggregation patterns and improved predictive performance relative to a standard conjoint model. Importantly, consumer segments defined by these aggregation patterns exhibit different demand curves and willingness to pay for product enhancements.
The Good News: Our brilliant audio engineers developed 12 incredibly cool features for our newest product lines! The Bad News: We can’t include them all. Now what? This talk shares how we used anchored MaxDiff, choice modeling, and elbow grease to identify feature subsets that balanced consumer reach with development constraints… and then dug deeper when the “best” solution raised more questions than it answered. The Result: An unexpected portfolio discovery, complete with musical montage!
Quality aesthetic design is essential to any product. However, aesthetics remain difficult to quantitatively measure, and therefore preference for different aesthetics can be difficult to capture in discrete choice tasks. Utilizing manipulations in a residual embedding space spanning 500 product categories and 2,710 adjectives, we introduce a workflow that allows for quantitative measurement of abstract aesthetic properties. This allows for concrete measurement of aesthetic preference in conjoint analysis and other discrete choice tasks.
Customers do not define value in the same way. This case study demonstrates how MaxDiff, Latent Class Cluster Analysis, and TURF were integrated to move from overall attribute rankings to four distinct needs-based segments and optimized value propositions for each. Attendees will learn how individual-level preferences uncover hidden market heterogeneity, why the best total-market bundle may not fit every segment, and how segment-level reach analysis translates advanced modelling into practical business positioning and activation decisions.
Compound decisions are determined by multiple, analytically separable components that operate simultaneously at the point of choice. Conjoint analysis changes when marginal utility is component-specific and sub-decisions cannot be tractably aggregated into a single constrained utility maximization problem with a common price sensitivity. We propose a new model for conjoint analysis whenever consumers can independently select the core product and the channel by which they gain access, and both exist in competitive markets.
As AI-led probing moves deeper into research, every follow-up promises richer understanding - and carries the risk of changing the very choices we want to measure. This study explores where useful explanation ends and intervention begins, testing how probe depth affects information gain, respondent behaviour and predictive accuracy. The answer may redefine not just how much AI should ask, but when it should safely stop.
The standard HB estimation of conjoint respondent utilities assumes a multivariate normal prior with unknown mean vector µ and covariance Σ as additional parameters. We describe a low-rank approximation Σ ~ LR*LR’:
- Σ is dimension PxP
- LR is reduced dimension PxPFac (recommending PFac<=10)
- For each respondent only PFac z-values are directly estimated, computing full P utilities as µ + LR*z.
Compared with estimations of full Σ using Inverse Wishart, Wishart, or LKJ, reduced factor parameterization decreases noise and improves speed, convergence, and holdout fit -- especially for large parameter spaces.
This session shows a browser-based, AI-built simulator that empowers clients to drive many of the most interesting analyses of choice data themselves: source-of-share investigation, optimizations, adopter profiling, price curves, feature sensitivity, calibrations, performance validations, and alternative choice rules as live controls, no server or license required. We pair the capability tour with the build-and validate practices that let a research team without coders ship such a tool safely.
Can six simple pricing questions replicate a robust conjoint study? Inspired by Bryan Orme's discussions of Van Westendorp and conjoint, we tested the question with 15,000+ consumers across 13 countries. Comparing CBC with Van Westendorp plus Newton-Miller-Smith reveals differences in demand-curve shape, implied price elasticity, and revenue-maximizing prices. We show why calibrating purchase likelihood at one price cannot correct differences in price sensitivity—and when those differences are large enough to change the recommended price.
AI-moderated interviewing is bringing qualitative depth to quantitative scale, but can MaxDiff travel with it? This study first tests standard MaxDiff across Web, text chat, WhatsApp, phone, real-time voice and video. A second experiment then asks a broader design question: what kind of MaxDiff works best in each AI-moderated environment? By varying task structure and respondent load, we explore how much MaxDiff each mode can support before measurement quality begins to suffer.
Many familiar survey design challenges, from respondent fatigue and task complexity to crowded conjoint exercises, reflect well-established principles of human cognition. This presentation applies a cognitive psychology lens to quantitative market research, connecting findings from MaxDiff, conjoint, and survey research with what we know about perception, attention, working-memory, and decision-making. Attendees will learn how these connections can inform design decisions and help anticipate where attention limits, working-memory constraints, and decision shortcuts may influence respondent behavior.
Hierarchical Bayes is the standard for individual-level utilities, but it is often too slow to test the modeling decisions that go into it. This session shows how Empirical Bayes, a fast alternative of HB, can search model specifications, tune estimation settings, and warm-start HB to cut burn-in. Across completed CBC and MaxDiff studies, we report where EB-based decisions transfer to full HB, where they do not, and how much project time this approach can save.
Synthetic data remains one of the most talked-about areas in market research. There are enthusiastic advocates, equally vocal sceptics, and an ever-growing body of technical research. But for practitioners trying to run an actual synthetic project, a more immediate question remains: how do you do it well? Drawing on our experience of delivering synthetic boosting and Digital Twin projects, this practical masterclass moves beyond the theoretical debate to focus on what happens when synthetic methods meet real research programmes, real data and real clients.
Our standard Hierarchical Bayes (HB) models rely only on respondent-level choices during estimation. However, in practice, conjoint results are primarily used to predict aggregate or segment-level choice shares. We explore a modification in which a semi-aggregate choice-share likelihood term is added to the HB likelihood. Early results suggest that this can improve aggregate share predictions while preserving the benefits of respondent-level estimation. This approach may also reduce the need for post-hoc adjustments such as scale corrections.
Track 1: Morning presenters
Track 2: Afternoon presenters
Track 3: Dave Lyon, Tom Eagle
Learn how you can use your own AI tools to interact with Lighthouse Studio via our MCP. Script, Test, Launch, Review and Analyze all with a single prompt to your LLM of choice!
Out-of-sample (OOS) predictive validation is widely recognized as the gold standard for evaluating CBC models. Sawtooth Software has developed a new OOS Validator that can perform automated k-fold OOS validation using HB-MNL estimation on virtually any CBC dataset.
In this presentation, Bryan will share findings from analyses of more than 30 CBC studies using the OOS Validator. Topics will include the impact of covariates in HB estimation, interaction effects, and utility constraints on predictive performance.
Do latent class segments used as covariates create a "double-dipping" problem? Do utility constraints improve realism at the expense of OOS predictive validity? Join us for answers to these questions and other practical insights from large-scale validation testing.
Can Qual-at-scale replicate, or perhaps improve upon, quantitative scales as an input into segmentation? Most segmentations run on a long attitudinal battery written before anyone speaks to a customer. We are testing whether a handful of open-ended questions, converted to embeddings, can build a better one. Same respondents, both instruments, three competing solutions, scored on holdout prediction, typing accuracy, reproducibility, interpretability, and interview length. The result tells the industry what segmentation inputs should be made of, and how much questionnaire we can give back to respondents.
Large language models increasingly shape how consumers discover, compare, and choose brands. This validation study asks whether AI Brand Visibility can be measured reliably enough to serve as a useful market-research metric. Controlled experiments test how results vary with prompt wording, user profiles, AI Surfaces (e.g. Google AI Mode) and time. We assess face validity, compare AI recommendations with consumer perceptions and sales data, and, where feasible, benchmark results against consumer-AI interactions observed through a respondent panel for validation.
Token-based conjoint helps researchers understand how bundles gain appeal as benefits accumulate. But do rising take rates mean each added benefit creates value? Building on an applied study, this presentation explores ways to separate baseline product appeal from the incremental value of benefits and account for bundle size. We will examine potential diminishing returns and discuss implications for modeling, simulation, and future research, including how to test larger bundles and identify where value may plateau.
Transformer models (e.g. LLMs) have revolutionized many aspects of life. However, progress in tabular data has been slow. In this talk I introduce Tabular Foundation Models, a class of transformer models designed and trained to excel in sparse tabular data. I introduce the models and benchmark their performance in classification and regression against established models. Further, I provide real-world evidence for when they are a substantial improvement and give advice on how to implement them.
Respondent covariates can make hierarchical Bayes choice models look better in sample while predicting entirely new respondents worse. We use a controlled stress test to show how covariates can induce severe overfit, where that overfit appears within the hierarchical model, and how covariate-level regularization can mitigate it. Applied cases show that regularization need not simply flatten meaningful respondent differences. We offer practical guidance for alleviating the downsides of respondent covariates in HB estimation in practice.
