Thematic Analysis & Questionnaires

Can thematic analysis be used for questionnaires?

Yes, thematic analysis can be a valuable tool for exploring qualitative data obtained through questionnaires. It works best on open-ended questions, where respondents explain their experiences in their own words.

Considerations When Using Thematic Analysis with Questionnaires

Questionnaire Design for Thematic Analysis

Virginia Braun sees real potential in surveys. She notes that they can gather rich and complex data, and can be a useful tool for qualitative research, even on sensitive topics.

Braun emphasizes that the open-ended format of qualitative surveys allows for unexpected responses and participant-driven narratives.

Braun et al. (2021) note that qualitative surveys remain under-used. They suggest two reasons: the dominance of interviewing, and a misplaced assumption that survey data lack depth.

Their paper counters that view with an overview and practical design guidance, drawn from the authors’ own online surveys.

This aligns with the principles of reflexive thematic analysis. It is Braun and Clarke’s own approach, with no fixed codebook, so codes and themes develop as the analysis deepens.

The researcher generates themes from respondents’ own words, rather than imposing predetermined categories. Themes do not simply emerge, as if waiting to be discovered. The researcher actively produces them (Braun & Clarke, 2019).

To get data suitable for thematic analysis, questionnaires must include open-ended questions that encourage detailed responses.

Such questions let participants describe their thoughts, experiences, and perspectives in their own words.

Researchers should write these questions clearly and concisely, and keep them focused on the research question.

Can thematic analysis be used with closed questions?

Closed-ended questions, such as multiple-choice or Likert scale questions, might not provide the depth of information needed for the nuanced interpretation that characterizes thematic analysis.

Closed questions tend to produce ‘bucket themes’. A bucket theme is a pre-defined topic area, often mirroring the questions asked, that groups everything respondents said about one subject. It names a topic without capturing any shared meaning.

Braun and Clarke (2019, 2021a) treat this as the most common failing in thematic analysis. They contrast these with shared-meaning themes, built around a central organizing concept. That concept is the idea that ties a cluster of codes together. Codes are short labels attached to pieces of data.

The difference shows in how a theme is named:

• Bucket theme: ‘Views on homework’ names a subject, not a shared meaning.
• Shared-meaning theme: “You’re never really off”: the collapse of the work–home boundary states an idea that runs through many answers.

Strong theme names usually contain a claim or a vivid phrase from a participant (Braun & Clarke, 2021a).

Context matters. Thematic analysis depends on understanding what shapes participants’ responses.

Closed-ended questions, especially in questionnaires, often lack the contextual richness that emerges from open-ended questions or interviews.

However, there are potential applications for thematic analysis in conjunction with closed questions:

• Analyzing open-ended responses within a primarily quantitative questionnaire: If a questionnaire includes a mix of closed-ended and open-ended questions, thematic analysis could be applied to the open-ended responses to explore the nuances of participants’ perspectives within the broader quantitative data set.
  • Content analysis suits open-ended answers when the aim is to count manifest content (surface wording) rather than interpret deeper meaning.
• Supplementing quantitative analysis with thematic insights: Researchers could consider a mixed methods approach. Closed questions show how widespread an experience is; thematic analysis of open responses or interviews helps explain why.
  • Thematic analysis of open-ended responses could provide a qualitative layer of understanding to complement the quantitative analysis of closed-ended questions.
• Exploratory phase of research: In the initial stages of research, closed-ended questions could be used to gather broad insights and identify potential areas for further exploration.
  • Subsequent qualitative methods, like interviews, could then be used to delve deeper into these areas, with thematic analysis applied to the qualitative data.

Data Volume and Saturation

The richness of the qualitative data obtained from the questionnaire directly impacts the quality of the thematic analysis.

If responses are brief or superficial, robust themes are hard to develop. A one-word answer gives the analyst little to code.

Just as with other qualitative data collection methods, researchers need to decide when they have enough data. The traditional benchmark is data saturation.

Data saturation means that collecting more data does not yield new themes or insights.

Reflexive thematic analysis treats saturation differently. Braun and Clarke (2021b) argue that saturation fits it poorly. If themes are generated rather than discovered, there is no fixed quantity of meaning waiting to be used up.

They prefer theoretical sufficiency. The analyst keeps coding and developing themes until they can give a rich, coherent answer to the research question.

Malterud et al. (2016) offer another route, called information power. The narrower the study aim, the more specific the sample, and the more existing theory the study draws on, the fewer participants it needs.

The same logic applies to questionnaires. Justify the number of respondents by the study’s purpose and the richness of their answers, not by a saturation point.

Piloting questionnaires is essential. It shows whether they generate rich data for thematic analysis.

Through piloting, researchers can test whether questions elicit detailed responses, identify unclear language, assess question flow, and refine prompts.

Piloting optimizes the questionnaire before full implementation, so the final data are deep enough for meaningful analysis. Fix weak prompts early.

Questionnaires can generate a large volume of data. Manual analysis can be slow.

Researchers might consider using software to assist with data management and coding. Some popular software options for qualitative data analysis include ATLAS.ti, Dedoose, MAXQDA, and NVivo.

Limited Context in Questionnaire Data

While questionnaires provide insights into participants’ perspectives, they might offer less contextual detail compared to interviews.

Context refers to the social, cultural, historical, and political factors that shape participants’ experiences and perspectives.

This context can include the setting where data was collected, the participant’s demographic characteristics, the power dynamics between the researcher and participant, and broader societal influences.

For example, a hospital setting may make participants more reserved. Demographic characteristics like race, gender, and socioeconomic status can also significantly influence their experiences.

This is a challenge for thematic analysis. Context is crucial for understanding the nuances of participants’ responses and developing comprehensive interpretations.

Unlike interviews, where researchers can probe for clarification or explore emerging ideas, questionnaires offer limited opportunity for dynamic interaction.

Researchers must read written responses closely for nuance and ambiguity. They cannot ask what a respondent meant.

Context also affects how deeply you can interpret an answer. Braun and Clarke (2006) distinguish two levels of meaning.

• Semantic themes stay close to what respondents explicitly wrote. ‘I never see my colleagues now’ becomes ‘reduced contact with colleagues’.
• Latent themes go beneath the surface to underlying ideas and assumptions. The same line might signal a felt loss of belonging that the respondent never stated.

With only a sentence or two per answer, there is often less context to support a latent reading.

While questionnaires may not capture context as naturally as interviews, researchers can proactively incorporate contextual questions within the questionnaire design:

• Demographic data, such as age, gender, ethnicity, socioeconomic status, and educational background, can provide valuable insights into participants’ social positions and how these might influence their perspectives
• Questions about the setting or circumstances surrounding the topic can also provide context. For example, if the questionnaire explores work-life balance, questions about work demands, family responsibilities, and social support systems could illuminate the context shaping participants’ responses.
• Including open-ended questions that encourage participants to elaborate on their experiences and provide their own contextual details is crucial.

Researcher bias is a real risk in thematic analysis.

Reflexivity involves critically examining your own assumptions and how they might influence the interpretation of the data.

Keep an audit trail. This is a recorded, decision-by-decision account of how you coded and built themes. It lets readers judge your interpretation for themselves (Nowell et al., 2017).

Member checking means returning to participants to confirm that their words were not misread. Anonymous questionnaires often make this impossible. Then the write-up should spell out exactly how responses were coded and grouped, and admit that the themes may not fully capture each respondent’s experience.

Deductive or Inductive Approach for Questionnaire Data

The choice between a deductive or inductive approach depends on the research aims and the nature of the questionnaire.

In practice, most analyses blend the two. A broadly inductive study still cannot escape the researcher’s disciplinary knowledge, and a broadly deductive study still notices things outside its frame (Braun & Clarke, 2006).

Deductive Thematic Analysis

A deductive approach to thematic analysis suits questionnaires designed to test existing theories or hypotheses. It also suits researchers who have pre-defined themes from a literature review or previous research.

Here the researcher starts with a theoretical framework. It guides the coding (labelling pieces of data) and theme development (grouping codes into broader patterns of meaning).

For example, a study investigating factors influencing people’s decisions to undergo genetic testing might use pre-existing theories about health decision-making as a starting point for their analysis.

Inductive Thematic Analysis

An inductive approach (e.g. reflexive thematic analysis) is more appropriate when the aim is to explore the data openly. Codes come directly from respondents’ own words.

This approach is particularly useful when investigating an under-researched area or working with participants whose views on the topic are not well-known.

In this approach, the researcher starts with the data and builds themes through a process of coding and theme development. Even so, themes do not emerge by themselves. The researcher actively generates them (Braun & Clarke, 2019).

For example, a researcher might use an inductive approach to analyze responses to a questionnaire exploring men’s experiences of body hair removal.

How to Analyze Open-Ended Questionnaire Responses

Braun and Clarke (2006) describe six phases of thematic analysis. The phases are recursive: you move back and forth between them, refining as you go.

1. Familiarize yourself: Read and re-read every response, noting early impressions and possible patterns. No formal coding happens yet.
2. Generate initial codes: Work through the whole dataset, attaching a short label to every segment relevant to the research question. One answer can carry more than one code.
3. Generate themes: Cluster related codes around a central organizing concept, using a thematic map (a diagram of how codes group) if helpful.
4. Review themes: Test each theme against its own extracts, then against the whole dataset. Discard, merge or redraw themes the data do not support.
5. Define and name themes: State what each theme is and is not, and give it an informative name, ideally with a vivid respondent phrase.
6. Write up: Weave the themes into a narrative that answers the research question. Quote several respondents, and discuss contradictory answers instead of smoothing them away.

A hypothetical example shows the phases at work. Imagine one open-ended question: ‘How has working from home affected you?’

In phase 2, each answer receives one or more codes:

• ‘I answer emails at 10 p.m. because the laptop is right there’: always-on availability
• ‘I never see my colleagues now’: reduced social contact
• ‘I like my boss but I prefer working at home’: positive relationship with manager, and preference for home working

In phase 3, the always-on codes cluster into a candidate theme, ‘The disappearing boundary between work and home’. The social-contact codes form a second, ‘Isolation and the loss of the workplace community’.

In phase 4, a tentative theme about technology problems rests on only two passing remarks. It is discarded.

In phase 5, the first theme gets a stronger name. It carries the theme’s meaning and a respondent’s own words: ‘My kitchen table is now my office’: the collapse of the work–home boundary.

Good reports name the version of thematic analysis they used. Reflexive, codebook and coding-reliability versions rest on different assumptions about quality, so naming yours lets readers judge the work by the right standard.

References

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4), 589–597. https://doi.org/10.1080/2159676X.2019.1628806

Braun, V., & Clarke, V. (2021a). One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative Research in Psychology, 18(3), 328–352. https://doi.org/10.1080/14780887.2020.1769238

Braun, V., & Clarke, V. (2021b). To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales. Qualitative Research in Sport, Exercise and Health, 13(2), 201–216. https://doi.org/10.1080/2159676X.2019.1704846

Braun, V., Clarke, V., Boulton, E., Davey, L., & McEvoy, C. (2021). The online survey as a qualitative research tool. International Journal of Social Research Methodology, 24(6), 641–654. https://doi.org/10.1080/13645579.2020.1805550

Malterud, K., Siersma, V. D., & Guassora, A. D. (2016). Sample size in qualitative interview studies: Guided by information power. Qualitative Health Research, 26(13), 1753–1760. https://doi.org/10.1177/1049732315617444

Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1), Article 1609406917733847. https://doi.org/10.1177/1609406917733847