Testing LLM-enhanced healthcare surveys with pregnant momsTesting LLM-enhanced healthcare surveys with pregnant moms
Testing LLM-enhanced healthcare surveys with pregnant momsTesting LLM-enhanced healthcare surveys with pregnant moms

Carien Moolman
We designed 4 surveys typically sent to pregnant moms in MomConnect via WhatsApp and enhanced these with an LLM. In July of this year we did one-on-one in-person qualitative usability testing of these surveys as well as their statically designed counterparts with 17 pregnant moms in Khayelitsha, Cape Town. Each mom sat with a single facilitator and was asked to complete a survey twice: first the static version, and then the LLM-enhanced version, while facilitators observed and made notes. Right after completion, the moms were asked which one they preferred and why they preferred that one. We accounted for ordering effects by asking half of our moms to begin with the static version, and the other half to begin with the LLM-enhanced version. The surveys we tested were: - a demographic information survey - a 5-question decision-making ability assessment - a knowledge, attitudes, and behaviour assessment regarding pregnancy - a short survey typically sent after an antenatal clinic appointment We also tested a feature where moms could ask any pregnancy health-related questions, as freeform text, and get an answer from the LLM, where the LLM was limited to approved healthcare information provided by our content team. Our findings were fascinating and point to: (1) the risks of using LLM-enhancement and how this might affect the credibility of data gathered (2) the need to keep humans in the loop on both the design side and the data moderation side of these kinds of studies (3) the immense potential of LLM-enhancement in the provision of digital healthcare information and services
About
Featured Posts
About
Featured Posts
About
Featured Posts
Talks from Day 1
Talks from Day 2
Jan 29, 2026
/
Post by
Theresa Rose
Amy Tayler
Doing what AI Can't: A Case Study in Digital Transformation
Doing what AI Can't: A Case Study in Digital Transformation
Jan 29, 2026
/
Post by
Leigh-Ann Clarke
Kristen Patterson
Designing Trust in Fintech: How UX Can Empower Financial Freedom
Designing Trust in Fintech: How UX Can Empower Financial Freedom
Jan 29, 2026
/
Post by
Alfi Oloo
Continuous Discovery in practice
Continuous Discovery in practice
Jan 29, 2026
/
by
Theresa Rose
Amy Tayler
Doing what AI Can't: A Case Study in Digital Transformation
Doing what AI Can't: A Case Study in Digital Transformation
Jan 29, 2026
/
by
Leigh-Ann Clarke
Kristen Patterson
Designing Trust in Fintech: How UX Can Empower Financial Freedom
Designing Trust in Fintech: How UX Can Empower Financial Freedom
Jan 29, 2026
/
Post by
Theresa Rose
Amy Tayler
Doing what AI Can't: A Case Study in Digital Transformation
Doing what AI Can't: A Case Study in Digital Transformation
Jan 29, 2026
/
Post by
Leigh-Ann Clarke
Kristen Patterson
Designing Trust in Fintech: How UX Can Empower Financial Freedom
Designing Trust in Fintech: How UX Can Empower Financial Freedom














