Pattern #24 · Research methods

Let the model cluster the transcript, keep the human naming the themes

Sorting quotes is clerical work. Deciding what they mean is the job.

Track Aux researchqualitative analysisthematic analysisllmresearch ops

Do

Hand the model the clerical middle: candidate codes, clusters, and pulling supporting quotes. Keep the codebook and the theme names human, and memo which steps the model touched.

Don't

Ask for the five themes. You get five confident headings, and five different ones tomorrow.

The rule. Use the model to group and retrieve qualitative data, and keep the codebook, the disagreements, and the theme names with the researcher, on the record.

Why. Goyanes, Lopezosa, and Jordá (2025) ran a six step ChatGPT protocol over 30 interviews: the model sped up exploration and reliably surfaced thematic patterns, but it struggled to interrelate those themes into a cohesive narrative, and the authors conclude it cannot substitute the contextual insight and reflexivity of human analysis. Nguyen-Trung (2025) guided GPT-4 through a four stage template analysis and found the same shape: capable at generating codes, subcodes, and clusters, with inconsistent outputs that required repeated prompt attempts. His framework keeps the researcher as the reflexive instrument and intellectual leader for exactly that reason.

Seen in the wild. ATLAS.ti's help center files AI generated codes under suggestions, lists "expecting AI Coding to replace analysis" as a common mistake, and tells researchers to memo every validation decision (verified as of September 3, 2026).

References

  1. 01

    ATLAS.ti. (2026, June 6). How to validate AI-generated codes in your analysis. ATLAS.ti Help Center. https://atlastihelp.helpscoutdocs.com/article/773-how-to-validate-ai-generated-codes-in-your-analysis

    https://atlastihelp.helpscoutdocs.com/article/773-how-to-validate-ai-generated-codes-in-your-analysis
  2. 02

    Goyanes, M., Lopezosa, C., & Jordá, B. (2025). Thematic analysis of interview data with ChatGPT: Designing and testing a reliable research protocol for qualitative research. Quality & Quantity, 59(6), 5491-5510. https://doi.org/10.1007/s11135-025-02199-3

    https://doi.org/10.1007/s11135-025-02199-3
  3. 03

    Nguyen-Trung, K. (2025). ChatGPT in thematic analysis: Can AI become a research assistant in qualitative research? Quality & Quantity, 59(6), 4945-4978. https://doi.org/10.1007/s11135-025-02165-z

    https://doi.org/10.1007/s11135-025-02165-z