Walk away with techniques you'll use next week

We outline a practical framework for improving online survey data quality by integrating behavioral, device, and historical feedback signals. Traditional checks like RLH risk discarding genuine respondents or retaining fraudulent ones, undermining insights. The research demonstrates how combining in-survey behavioral indicators, device fraud detection, and cross-platform feedback loops creates a dynamic, adaptive quality management system. By leveraging Data Quality Co-op's API within Lighthouse Studio, the study highlights holistic, iterative methods for identifying fraud and preserving authentic consumer voices.

While MaxDiff excels at selecting the best product claims, it falls short for evaluating longer technical narratives. We hypothesized that limited consumer awareness of our composite lumber's advantages over industry standards was hindering adoption. Using a vignette-based survey design, this study tested behavioral shifts after exposure to a technical product story. Results demonstrated that incorporating educational content into our marketing strategy boosts conversion 80%. Vignettes are a promising addition to incorporate into mixed-method surveys.

For B2B SaaS companies, growth requires moving beyond intuition-based pricing. This paper presents an integrated case study with Docusign, showcasing a multi-stage conjoint research program that tackled distinct strategic challenges: launching a new product line, managing cannibalization with license types, creating tiered service offerings, and channel expansion. Attendees will gain a practical framework for using advanced choice modeling to drive customer-centric monetization and translate complex research into measurable revenue growth and clear go-to-market guidance.

Creating a banking product that truly shines takes more than intuition - it takes the right ingredients, smart tools, and a bit of creative flair. Bank Millennium faced the challenge of designing a new offer amid trillions of possible value proposition combinations. By blending qualitative insights with Adaptive Choice-Based Conjoint, we discovered the mix of features that both customers value and the business can profitably deliver.

In Venezuela's uniquely complex market and economic environment, Empresas Polar adapted choice-based studies to generate fast, cost-effective insights under extreme volatility. By developing in-house capabilities, the team refined segmentation, tracked price sensitivity, forecasted sales and informed pack-size, pricing, and innovation strategies. Attendees will learn how abandoning traditional paradigms and tailoring conjoint designs to local constraints yielded actionable results. The case provides replicable lessons for organizations navigating unstable markets, offering practical guidance for agile, resilient research.

Synthetic data promises faster, cheaper, and more flexible research—but it comes with risks. Using frameworks such as Truth, Transparency, and Trust, and blending Human and Artificial Intelligence, safe outcomes can be achieved. This paper explores the do's and don'ts of synthetic augmentation, then presents an empirical study with a Digital Twin Panel, where AI-generated "clones" of real respondents are tested in choice studies, examining whether digital twins can represent the next generation of research automation.

One way to employ AI technology in surveys is by personalizing respondent stimuli. In this session we’ll show how that works with hyper-personalization of marketing messages, turning typical message testing into an approach that follows the reality of digital targeting practices. Beyond message tests, we’ll cover practical tricks anyone can quickly pick up to implement AI into their survey workflows with just about any question format and platform. We’ll include a preview of upcoming tools soon to be found in Sawtooth's Discover survey platform that will make that process even easier.

Synthetic data is a polarizing topic, simultaneously touted as a solution for hard-to-reach populations and small-sample studies, and condemned as confidently misrepresenting the voice of the customer. In this session, we present a validation framework along with findings from our ongoing evaluation of synthetic data across common tasks including cross tabs, key driver analysis, and conjoint. Attendees will leave with practical guidance for evaluating synthetic data outcomes along with validated use cases.

Can large language models replace manual coding of open-end survey responses without sacrificing quality? We outline a four-phase approach: (1) testing AI-generated codeframes against human baselines; (2) evaluating assignment accuracy via interrater reliability and agreement subsets; (3) weighing cost-benefits of additional training data; and (4) sharing lessons from operationalizing at KS&R. Attendees will leave with methods for quantitatively assessing AI's utility in open-end coding and practical tools to begin experimenting within their own organizations.

To reduce survey length, researchers have respondents answer only a subset of questions. The missing data is then imputed using advanced data imputation techniques. We evaluate the effectiveness of LLMs' imputation capabilities against traditional methods, using real human responses as the benchmark. Our findings offer valuable insights into how and under what conditions LLMs can enhance data quality and efficiency in survey analysis.

Shapley Values (SVs) are widely used in summarizing item contributions and importance, but become computationally impossible for, say, 30-plus items. We will briefly introduce SVs and a trick for computation in variants of TURF, but focus on a new class of experimental designs that dramatically improve the accuracy of SV sampling approaches and make even 200 items feasible, for all problem types. We will also compare SVs to Johnson's Relative Weight Analysis for key driver regressions.

Get an in-depth look at the new Lighthouse Studio MaxDiff Simulator and how it can be used to explore realistic portfolio and “what-if” scenarios. This clinic will walk through recent simulator enhancements, including the introduction of Shapley Values as a principled way to understand item contributions and shared effects in MaxDiff results. We’ll discuss how Shapley-based attribution helps you better understand TURF results.

Conjoint analysis often begins with static feature glossaries, but are respondents really reading them? This research-on-research study compares glossary-based warm-ups with a Kano-inspired warm-up, with and without the Kurz-Binner Priming questions. We test impacts on comprehension, ANA, engagement, LOI, and utilities. Attendees will walk away with evidence-based, practical guidance for designing conjoint warm-ups that improve both the respondent experience and the quality of the data.

At the 2021 Turbo Choice Modeling Event, Peitz and Lerner introduced Bespoke CBC, where respondents choose which attributes matter to them, and only those appear in their CBC tasks. This personalization improved engagement, reduced dropout, and increased predictive validity—but was hard to program. We've developed a simple coding method that removes this barrier. Our next steps include empirically testing Bespoke CBC against other CBC types, examining attribute non-attendance, and assessing robustness when respondents misidentify key attributes.

This study examines whether differentiating outside good choices in discrete choice models, into channel switching, category substitution, and category exit, yield richer insights than treating it as a single no-choice option. Using plant-based dairy as a case study, we evaluate improvements in model fit, predictive validity, and stability of willingness-to-pay across consumer subgroups. By clarifying how consumers opt out, the approach enhances market realism and equips researchers with strategies to navigate demand shifts in competitive markets.

The growing presence of AI in market research has created both excitement and questions. In this session, we walk through a researcher’s “perfect day” and show how AI can make everyday tasks easier - programming surveys in Sawtooth, handling translations, probing and coding open-ends, analysing data, building simulators, and preparing reports. Using real project learnings and the Automate–Augment–Avoid approach, we explain where AI truly helps, where it needs your guidance, and where human thinking matters most. You’ll walk away with practical tips and an easy-to-use AI Readiness Index that clarifies how much AI to apply at each of these key workflow stages

Conjoint analysis often includes a no-choice option allowing respondents to opt out and not make a purchase. We evaluate a dual-response format in which respondents first select their preferred option excluding the no-choice alternative, then decide whether they would actually purchase it. Results show that respondents apply budget constraints only in the second response, separating affordability from preference. The dual-response model improves parameter estimation efficiency by 30% and corrects overestimated equilibrium prices in competitive market simulations.

Companies possess a significant amount of relevant data, but due to a lack of resources and time, they often experience difficulties in utilising it. We aim to demonstrate our LLM-based approach solution, in which LLM acts as a smart interface between human intent and complex and unorganized data enabling automated, scalable, and user-friendly processing of complex product characteristics. The presentation will show how we transformed a technical bottleneck into a flexible, collaborative workflow.

In Filter-CBC, respondents often fail to apply filters sufficiently, leaving them overloaded with irrelevant alternatives and increasing position bias. This study introduces pre-filtering modules before the conjoint exercise to reduce these effects. Two approaches are evaluated: an adaptive CBC pre-module and a simplified attribute-ranking with must-have probing. Both identify which attributes and levels to prefilter, thereby reducing task complexity and cognitive load, and ultimately leading to more reliable and predictive utility estimates.

Survey fraud today ranges from bots and device spoofing to inattentive, low-effort respondents - yet most tools protect only the entry point or the final dataset. This session reviews the data-quality landscape to show what current systems handle well and where gaps remain. We then demonstrate a multi-layer framework, implemented directly within Sawtooth, that uses real-time validation, behavioral signals, AI-led open-end quality checks, and respondent-level scoring to monitor integrity throughout the survey. The takeaway: a practical, replicable way to bring continuous data-quality protection into your Sawtooth studies - turning every complete into cleaner, more dependable insight.

Markets like video streaming services in which buyers purchase more than one good at the same time pose substantial problems to Conjoint practitioners. We present and discuss ways to properly model these choices. Based on real projects we show advantages and disadvantages of different approaches and guide users on how to make the most of the choice data that they collected.

Clients often request feature-level willingness to pay (WTP) insights, but additive methods face scaling and feasibility issues. Sawtooth's Sampling of Scenarios (SOS) improves realism through scenario averaging yet still overstates values. SKIM is testing alternative extrapolation functions (quadratic, exponential, and piecewise) to better capture price responses, with preliminary results showing a 16% WTP reduction using quadratic extrapolation versus linear. Future work also includes broader scaling approaches comparing "all-features-on" versus "all-features-off" simulations.

Traditional product development relies on expert-led ideation and evaluation, which is time-intensive and may overlook valuable concepts. We present an AI-augmented approach engaging consumers directly in co-creation using an AI-powered conversational survey agent. Participants generate benefit-feature ideas with chatbot assistance, followed by rolling MaxDiff evaluation. This method produces human-created ideas with quantitative scores, AI-enabled analysis, and rich qualitative insights, demonstrating AI's potential for augmenting human creativity in product development.

Generative AI offers a new way to improve conjoint analysis: by supplying utility priors for design and estimation. In this session, we show how large language models (LLMs) can generate realistic prior utilities that enhance predictive accuracy in CBC studies. Using a job search platform experiment with ~1,000 participants, we compare standard designs with GenAI-augmented ones and share practical lessons and pitfalls.

After fielding a conjoint or MaxDiff survey and spending time with the various analysis and simulation tools available, it is time to present results. This breakout session will review examples, industry best practices, and other tips for presenting and interpreting data from a choice exercise. Attendees will be provided a presentation template and answers to frequently asked questions from clients related to visualizing and interpreting conjoint and MaxDiff data.

Hear all about recent updates to the Discover platform. We recently released a huge update to conjoint exercises with design metrics, automated tests and sample size recommendations. We’ll also cover email distribution and research panel access, and end with a showcase of our experiments with AI and an open discussion about the merits and challenges of implementing language models into a SaaS product.

One of best ways to improve your chances of a successful segmentation study is to avoid using as bases variables that can harm your segmentation. In this clinic we'll quickly review some ways that choosing the wrong basis variables can damage your segmentation. Then we'll move on to cover some variable selection methods that remove harmful variables and enable more powerful segmentation with more differentiated segments.

Exercise-related biases in stated choice experiments, such as position effects and screen fatigue, can reduce model accuracy by introducing skewed responses. We evaluate two mitigation strategies: adding contextual information to utility models and adjusting pooling based on respondent behavior. Meta-analytic findings will assess their effectiveness across studies. Attendees will gain practical tools for identifying quasi-straightliners, cleaning noisy data, and applying recommendations to model, simulate, and improve the overall quality of choice experiment results.

Driver-CBC is an innovative approach that integrates brand association analysis with choice-based conjoint models to inform market decisions. This method captures the impact of brand perceptions on consumer choices, enabling simulations of various brand perceptions, price and product feature scenarios and their consequent economic outcomes. By combining tactical and strategic elements, Driver-CBC provides a unified framework for optimising marketing strategies and brand development, offering actionable insights to maximise market share and revenue growth.

Automated Machine Learning (AutoML) identifies and then executes the best algorithm fitting to each specific data set automatically with little or no human intervention. Expanding AutoML to clustering is gaining attention but facing additional hurdles due to the lack of ground-truth for model training. Making it work for market research is even more challenging because of the art part. But the value can be big. This research explores an automated system for market segmentation, shares learnings and novel solutions for future expansion.

After designing a conjoint survey and collecting your data, the next step is understanding how to translate the raw data into actionable insights — through simulations. This breakout session will guide attendees through building a basic Excel conjoint simulator from scratch and demonstrate how more advanced features can be included. Participants will see step-by-step examples, and learn tips for structuring and customizing simulators to answer real business questions.

In this talk I will describe several techniques for aligning questions from multiple surveys so that they can be integrated and used in models to predict segment membership. I will demonstrate the techniques and describe the necessary assumptions behind them. No AI is required.