Understanding Human–AI Workflows for Generating Personas
How to generate representative and empathy-evoking personas with LLMs?
Challenge: The high effort in working with user data often leads to generating personas that do not represent or evoke empathy toward users.
Motivation: Large language models (LLMs) could assist in summarizing user data into personas with their text summarization capabilities. However, they also have technical limitations such as biases that can taint the quality of personas.
Approach: We explored human-AI workflows for generating high-quality personas by differently delegating persona-generation subtasks to user researchers vs. LLMs.
Result: The most representative and empathy-evoking personas can arise when user researchers take the lead role in creating archetypal user groups and exploit LLMs' summarization capability ("LLM-summarizing" in the figure above).
How would LLMs transform the use of personas?
In our formative study, we found that the interaction via LLMs enables simulating user interview, further inquiring about users and user feedback on design solutions.
For more details, please check out our paper: Paper
Abstract
One barrier to deeper adoption of user-research methods is the amount of labor required to create high-quality representations of collected data. Trained user researchers need to analyze datasets and produce informative summaries pertaining to the original data. While Large Language Models (LLMs) could assist in generating summaries, they are known to hallucinate and produce biased responses. In this paper, we study human--AI workflows that differently delegate subtasks in user research between human experts and LLMs. Studying persona generation as our case, we found that LLMs are not good at capturing key characteristics of user data on their own. Better results are achieved when we leverage human skill in grouping user data by their key characteristics and exploit LLMs for summarizing pre-grouped data into personas. Personas generated via this collaborative approach can be more representative and empathy-evoking than ones generated by human experts or LLMs alone. We also found that LLMs could mimic generated personas and enable interaction with personas, thereby helping user researchers empathize with them. We conclude that LLMs, by facilitating the analysis of user data, may promote widespread application of qualitative methods in user research.