Advanced track

Advanced Preference Modelling

Don’t miss this opportunity to expand your skills and network with leading choice modelling practitioners. This advanced track will focus on techniques such as Menu based Choice, Situational Choice Experiments and ACBC, as well as having papers from guest speakers (Ipsos & GfK) focussing on brand premium analysis, SKU based Conjoint and the ability of LLMs ability to answer choice models. Experts from Sawtooth will also share some key findings from recent Conference and TURBO events. Also included is an hour-long expert panel discussion where you can learn from leading practitioners in the field.

Register now
Day 1

Monday, 7 October

8:30

Welcome Coffee

8:30

Welcome Coffee

9:00

Intro to Situational Choice

A situational choice experiment (SCE) differs from a CBC in that its questions show one experimentally designed profile which describes the attributes and levels of the choice situation or context or chooser, and elicits choices among a fixed set of alternatives; SCE then uses polytomous MNL to generate utilities. Few marketers know about SCEs and there doesn’t seem to be a single source reference about them, two limitations we hope to remedy in this presentation.

Keith Chrzan
Senior VP, Sawtooth Analytics
Dean Tindall
Managing Director, Sawtooth Software Europe

9:00

Intro to Situational Choice

A situational choice experiment (SCE) differs from a CBC in that its questions show one experimentally designed profile which describes the attributes and levels of the choice situation or context or chooser, and elicits choices among a fixed set of alternatives; SCE then uses polytomous MNL to generate utilities. Few marketers know about SCEs and there doesn’t seem to be a single source reference about them, two limitations we hope to remedy in this presentation.

Keith Chrzan
Senior VP, Sawtooth Analytics
Dean Tindall
Managing Director, Sawtooth Software Europe

11:00

Refreshment Break

11:00

Refreshment Break

11:30

Intro to Menu-Based Choice & Perceptual Choice

Menu-Based Choice (MBC) is a flexible choice modeling approach for solving a variety of multi-check (combinatorial) menu-selection problems. Examples include: choosing options to put on an automobile, selections from a restaurant menu, banking options, configuring an insurance policy, purchasing bundled vs. a la carte services including mobile phones, internet, and cable. This session will seek to introduce you to the concept, theory and practice of undertaking an MBC study, with a particualr focus on how this can be used to create Perpceptual Choice Experiments.

Bryan Orme
President & CEO, Sawtooth Software
Dean Tindall
Managing Director, Sawtooth Software Europe

11:30

Intro to Menu-Based Choice & Perceptual Choice

Menu-Based Choice (MBC) is a flexible choice modeling approach for solving a variety of multi-check (combinatorial) menu-selection problems. Examples include: choosing options to put on an automobile, selections from a restaurant menu, banking options, configuring an insurance policy, purchasing bundled vs. a la carte services including mobile phones, internet, and cable. This session will seek to introduce you to the concept, theory and practice of undertaking an MBC study, with a particualr focus on how this can be used to create Perpceptual Choice Experiments.

Bryan Orme
President & CEO, Sawtooth Software
Dean Tindall
Managing Director, Sawtooth Software Europe

13:00

Lunch

13:00

Lunch

14:30

A Practitioners Guide to MBC Projects

Researchers often encounter situations where the client wants to optimize their product configuration, but the product’s ultimate configuration is determined by the consumer. Menu based choice models can help researchers understand how consumers make related decisions in markets like food service, telecoms, and many other situations. The very nature of these projects requires more complex thinking than standard choice models so this practical talk will discuss the questions that you need to be asking the client, and tips and tricks for ensuring you get the best out of these studies.

Chris Moore
Head of Data Science Transversal, Ipsos

14:30

A Practitioners Guide to MBC Projects

Researchers often encounter situations where the client wants to optimize their product configuration, but the product’s ultimate configuration is determined by the consumer. Menu based choice models can help researchers understand how consumers make related decisions in markets like food service, telecoms, and many other situations. The very nature of these projects requires more complex thinking than standard choice models so this practical talk will discuss the questions that you need to be asking the client, and tips and tricks for ensuring you get the best out of these studies.

Chris Moore
Head of Data Science Transversal, Ipsos

15:00

TURBO - How Many CBC Questions

We use our TURBO event each year to run a series of experiments, helping to further enhance our understanding of choice modelling. In this session Keith will share his paper focussing on how many tasks is appropriate for CBC studies.

Keith Chrzan
Senior VP, Sawtooth Analytics

15:00

TURBO - How Many CBC Questions

We use our TURBO event each year to run a series of experiments, helping to further enhance our understanding of choice modelling. In this session Keith will share his paper focussing on how many tasks is appropriate for CBC studies.

Keith Chrzan
Senior VP, Sawtooth Analytics

15:30

TURBO - How Sparse Is Too Sparse in MaxDiff

Continuing our TURBO paper session, Dean will focus on the differences in error generated by sparse and express MaxDiff formats.

Dean Tindall
Managing Director, Sawtooth Software Europe

15:30

TURBO - How Sparse Is Too Sparse in MaxDiff

Continuing our TURBO paper session, Dean will focus on the differences in error generated by sparse and express MaxDiff formats.

Dean Tindall
Managing Director, Sawtooth Software Europe

16:00

Refreshment Break

16:00

Refreshment Break

16:30

TURBO - NOL Effect in CBC

In our final TURBO paper, Bryan will share his paper which investigated whether the number of levels effect is as scary as it might sound.

Bryan Orme
President & CEO, Sawtooth Software

16:30

TURBO - NOL Effect in CBC

In our final TURBO paper, Bryan will share his paper which investigated whether the number of levels effect is as scary as it might sound.

Bryan Orme
President & CEO, Sawtooth Software

17:00

Investigating the Use of Behavioural Questions to Improve CBC Predictions

Bryan will present a paper seeking to validate the findings from the Kurz/Binner 2021 Sawtooth Software Conference presentation that won best paper. Kurz/Binner demonstrated how a simple grid of 9 binary “Behavioral Calibration Questions” that probed how respondents regarded brand, innovation, and price could significantly improve the consistency of respondents’ CBC data and also their holdout predictive validity.

Bryan Orme
President & CEO, Sawtooth Software

17:00

Investigating the Use of Behavioural Questions to Improve CBC Predictions

Bryan will present a paper seeking to validate the findings from the Kurz/Binner 2021 Sawtooth Software Conference presentation that won best paper. Kurz/Binner demonstrated how a simple grid of 9 binary “Behavioral Calibration Questions” that probed how respondents regarded brand, innovation, and price could significantly improve the consistency of respondents’ CBC data and also their holdout predictive validity.

Bryan Orme
President & CEO, Sawtooth Software

17:30

Adjourned

17:30

Adjourned

Back to top
Day 2

Tuesday, 8 October

8:30

Welcome Coffee

8:30

Welcome Coffee

9:00

ACBC is a powerful technique for many reasons, however, some users still prefer the flexibility of CBC for their studies. Bryan will use this session to teach a series of power tricks which can open up the use of ACBC to more studies, including the use of constructed lists, summed pricing and many more advanced features.

Intro to Advanced ACBC

Bryan Orme
President & CEO, Sawtooth Software

9:00

Intro to Advanced ACBC

ACBC is a powerful technique for many reasons, however, some users still prefer the flexibility of CBC for their studies. Bryan will use this session to teach a series of power tricks which can open up the use of ACBC to more studies, including the use of constructed lists, summed pricing and many more advanced features.

Bryan Orme
President & CEO, Sawtooth Software

10:00

SKU-Based Conjoint involves presenting respondents with numerous products ‘Stock Keeping Units (SKUs)’ at different price points. In contrast to typical conjoints, the product concept is fixed and refers to existing (or close to launch) products. This design is best suited for pricing research. However, the complexity arises from the number of SKU, leading to lengthy surveys that can overwhelm respondents. This can result in decreased data quality and unreliable insights. The session will present an overview of previous foundational research in the area and provide recommendations for some of the best practices in areas such as selection bias, nested simulations, data augmentation and modelling the price function.

SKU-Based Conjoint

Manjula Bhudiya
Associate Director of Advanced Analytics, Ipsos

10:00

SKU-Based Conjoint

SKU-Based Conjoint involves presenting respondents with numerous products ‘Stock Keeping Units (SKUs)’ at different price points. In contrast to typical conjoints, the product concept is fixed and refers to existing (or close to launch) products. This design is best suited for pricing research. However, the complexity arises from the number of SKU, leading to lengthy surveys that can overwhelm respondents. This can result in decreased data quality and unreliable insights. The session will present an overview of previous foundational research in the area and provide recommendations for some of the best practices in areas such as selection bias, nested simulations, data augmentation and modelling the price function.

Manjula Bhudiya
Associate Director of Advanced Analytics, Ipsos

11:00

Refreshment Break

11:00

Refreshment Break

11:30

Traditional brand trackers focus on a brand’s ability to generate volume through measures of stated consideration and preference but often overlook the ability of a brand to charge a higher price. We show how conjoint analysis not only more accurately measures brand volumes but can also measure brand price premium. Furthermore, we show that what you need to do to build brand premium is distinctively different to how to drive brand volume. We delve into different ways of calculating price elasticity and examine the impact of displaying prices to respondents as proportional prices rather than monetary values. Given many brands invest significantly in brand tracking, this provides an exciting new opportunity for the application of conjoint analysis.

Supercharge Your Brand Health Tracker Using Conjoint Analysis

Alex Chirilov
Head of Global Marketing Science, GfK
James Pitcher
Marketing Sciences Lead UK, GfK

11:30

Supercharge Your Brand Health Tracker Using Conjoint Analysis

Traditional brand trackers focus on a brand’s ability to generate volume through measures of stated consideration and preference but often overlook the ability of a brand to charge a higher price. We show how conjoint analysis not only more accurately measures brand volumes but can also measure brand price premium. Furthermore, we show that what you need to do to build brand premium is distinctively different to how to drive brand volume. We delve into different ways of calculating price elasticity and examine the impact of displaying prices to respondents as proportional prices rather than monetary values. Given many brands invest significantly in brand tracking, this provides an exciting new opportunity for the application of conjoint analysis.

Alex Chirilov
Head of Global Marketing Science, GfK
James Pitcher
Marketing Sciences Lead UK, GfK

13:00

Lunch

13:00

Lunch

14:30

Judgement Day: The Machines Have Arrived – But How Good Are They at Answering Choice Experiments

This presentation investigates whether large language models (LLMs) can replicate human behaviour in answering both Conjoint and MaxDiff choice tasks. The research comprising of more than 50 experiments, reviewed different LLM’s, parameter settings such as temperature, and whether different executions of the prompts would yield better results. More than 250,000 LLM choices were generated to see how close LLM’s could get to replicating the utility structure of real-world Conjoint and MaxDiff projects and the presentation will answer eight key hypotheses that were tested. The big question is whether machines can replace human respondents or is there still a role for human respondents and analysts!

Chris Moore
Head of Data Science Transversal, Ipsos

14:30

Judgement Day: The Machines Have Arrived – But How Good Are They at Answering Choice Experiments

This presentation investigates whether large language models (LLMs) can replicate human behaviour in answering both Conjoint and MaxDiff choice tasks. The research comprising of more than 50 experiments, reviewed different LLM’s, parameter settings such as temperature, and whether different executions of the prompts would yield better results. More than 250,000 LLM choices were generated to see how close LLM’s could get to replicating the utility structure of real-world Conjoint and MaxDiff projects and the presentation will answer eight key hypotheses that were tested. The big question is whether machines can replace human respondents or is there still a role for human respondents and analysts!

Chris Moore
Head of Data Science Transversal, Ipsos

15:15

A&I Summit 2024 Recap

Bryan brings us a run down of the advances made from the best papers from the 2024 A&I Summit, held during May in San Antonio, Texas.

Bryan Orme
President & CEO, Sawtooth Software

15:15

A&I Summit 2024 Recap

Bryan brings us a run down of the advances made from the best papers from the 2024 A&I Summit, held during May in San Antonio, Texas.

Bryan Orme
President & CEO, Sawtooth Software

16:00

Refreshment Break

16:00

Refreshment Break

16:30

Ask the Experts (Panel Discussion)

Choice modelling experts from Ipsos, GfK & Sawtooth are on hand to discuss an array of topics as well as to answer any questions you might have on the world of choice modelling.

Bryan Orme
President & CEO, Sawtooth Software
Keith Chrzan
Senior VP, Sawtooth Analytics
Chris Moore
Head of Data Science Transversal, Ipsos
Alex Chirilov
Head of Global Marketing Science, GfK

16:30

Ask the Experts (Panel Discussion)

Choice modelling experts from Ipsos, GfK & Sawtooth are on hand to discuss an array of topics as well as to answer any questions you might have on the world of choice modelling.

Bryan Orme
President & CEO, Sawtooth Software
Keith Chrzan
Senior VP, Sawtooth Analytics
Chris Moore
Head of Data Science Transversal, Ipsos
Alex Chirilov
Head of Global Marketing Science, GfK
James Pitcher
Marketing Sciences Lead UK, GfK
Dean Tindall
Managing Director, Sawtooth Software Europe

17:30

Adjourned

17:30

Adjourned

Back to top
DON’T MISS OUT

Save your spot today!