When it comes to online surveys, we need to trust that respondents understand and interpret the tasks the way we expect them to. To investigate this, we conduct a series of qualitative interviews aimed to observe how respondents answer, understand and perceive conjoint exercises. Our findings, among others, show that instructions are crucial but often too lengthy, choices depend on anchoring and choices are driven by variety seeking or opportunity to trial new things.