Surveys can feel fast and cheap, but used badly, they entrench bias and stall discovery.
Surveys have long been a popular tool for gathering user insights in UX research, but they come with inherent risks that can easily skew results and mislead decision-makers. In her WebExpo 2025 talk, Lead UX Researcher Zsuzsa Kovács focused on the critical dangers associated with surveys and offered practical advice on when and how to employ them effectively.
The allure and risks of surveys
Zsuzsa began by acknowledging the pervasive presence of surveys, noting their ease of access and low cost, which make them attractive to companies seeking quick feedback. However, she warned that “you might be thinking you are doing research when in fact you might be only confirming your preconceptions without getting real insights from users.” This highlights the dangers of using surveys without the necessary rigour or understanding, potentially leading to biased results that reinforce false assumptions.
Challenges with survey data
A significant concern raised by Zsuzsa is the challenge of obtaining a representative sample of respondents. She explained that “fans and haters are the most likely to answer,” referring to individuals who either love or despise the product or service. Such extreme responses often drown out valuable, objective feedback from more neutral users. Furthermore, respondents may not thoroughly read or consider the survey questions, leading to superficial or inaccurate data collection.
Zsuzsa also pointed out several cognitive biases that can affect surveys, including satisficing, where respondents choose the first seemingly adequate answer rather than the best one. She described, “It’s very easy to put together a survey that will only bring you back the things you wanted to hear,” underscoring the pitfalls of poorly constructed survey questions.
Designing better surveys
Despite her critique, Zsuzsa does not dismiss surveys altogether. Instead, she shared ways to make them work:
- Add open questions which help expose blind spots
- Keep wording and response options simple
- Use them to quantify attitudes, not behaviours, since people are unreliable at reporting what they actually do
And one golden rule: “Never start discovery with surveys. That’s a very, very bad thing to do!” For behavioural insight, usability tests will always beat self-reporting.
The trap of leading questions
To prove her point, Zsuzsa ran a live poll during her talk. She asked: “The climate catastrophe is defining our world. Do you try to use your car less to save the environment?” The audience quickly saw how such wording nudges people toward a “correct” answer. “It’s leading and you know which answer is expected,” she said.
Smarter alternatives
Instead of traditional surveys, Zsuzsa suggested contextual micro-surveys. Ask about an experience right after it happens. For example, “Registration process? Ask about it right after they complete it,” she suggested. This approach minimises recall biases and enhances the reliability of the data collected.
She also took aim at the beloved Net Promoter Score (NPS), labelling it as “totally worthless” because of cultural bias and reliance on hypotheticals. Instead, Zsuzsa advocated for creating surveys with specific goals in mind and mixing in other research methods for a fuller picture.
Expert advice and further learning
For anyone looking to deepen their practice, Zsuzsa recommended Erika Hall and Caroline Jarrett, both ex-WebExpo speakers and long-time champions of better research. Their books and articles are packed with practical wisdom.
She closed with advice worth sticking above your desk: “Read articles, get feedback from peers, and always iterate on your surveys.”
Want to hear it straight from Zsuzsa? Watch the full recording along with her slides right here: