An audit trail in qualitative research is a systematic and comprehensive record of the research process that enables an external reviewer to trace the study’s path and assess the dependability and confirmability of the findings.

The researcher should document their decisions, procedures, and interpretations throughout the study.

This includes information about data collection, analysis, and any changes made to the research design.

Audit trails promote transparency by making the researcher’s decisions, procedures, and interpretations visible and traceable.

The audit trail serves several purposes:

Transparency: It makes the research process transparent, allowing others to understand how the study was conducted and how conclusions were reached.
Accountability: It holds the researcher accountable for their decisions and interpretations.
Verification: It enables others to verify the study’s findings by retracing the steps taken.
Credibility Enhancement: A well-documented audit trail strengthens the credibility of the research by demonstrating a systematic and rigorous approach.

What should be included in an audit trail?

Records should be detailed enough to allow an external reviewer to understand the decisions made and the reasoning behind them.

The audit trail is not merely a final product; it’s an ongoing process. It should be initiated at the beginning of the research and meticulously maintained throughout the study.

Lincoln and Guba (1985, p. 319-310) cite Halpern’s (1983) categories for reporting information when develop an audit trail.

Raw data: Field notes, interview transcripts, audio/video recordings, and documents collected.
Data reduction and analysis products: Codes, memos, summaries, and working hypotheses.
Data reconstruction and synthesis products: Thematic categories, interpretations, and inferences.
Process notes: Methodological decisions, challenges encountered, and reflections on the research process.
Materials related to intentions and dispositions: Research proposal, field journal, and reflexivity notes.
Instrument development information: Pilot forms, surveys, and interview guides.

Example

The key is to create a comprehensive, well-organized, and easily accessible record that illuminates the research process and bolsters the trustworthiness of the findings.

Project Title: Understanding the Lived Experiences of Individuals Caring for Family Members with Dementia

Researcher: [Your Name]

Date: [Date]

1. Research Design and Proposal

Initial Research Proposal: (Date) Includes research question, rationale for the study, proposed methodology (e.g., phenomenology), and a preliminary interview guide.
Ethics Approval Documentation: (Date) Documentation of ethics approval obtained from the relevant institutional review board.
Literature Review Summary: (Date) A synthesis of key literature informing the study’s theoretical framework and research questions.

2. Data Collection

Participant Recruitment Materials: (Date) Copies of recruitment flyers, advertisements, or emails used to recruit participants.
Informed Consent Forms: (Date) Templates and signed consent forms from each participant.
Interview Guide: (Date) The final version of the interview guide used, along with notes on any modifications made during data collection.
Field Journal: (Date) A chronological record of the researcher’s observations, reflections, and methodological decisions made during data collection. This journal could also include notes on challenges encountered and solutions implemented. For example, a researcher might note difficulties in recruiting participants from a specific demographic and describe how they adapted their recruitment strategies.
Audio Recordings and Transcripts: (Date) Digital audio recordings of each interview, along with corresponding transcripts. Important considerations for transcripts:
- Level of Detail: The level of detail in the transcripts (verbatim or edited for clarity) and the rationale for this choice should be documented.
- Transcription Conventions: A codebook outlining the transcription conventions used (e.g., notation for pauses, overlapping speech, or nonverbal cues) should be included.
- Accuracy Checks: Documentation of steps taken to ensure the accuracy of the transcripts, such as comparing the transcript to the audio recording and addressing any discrepancies.
Data Management Log: (Date) A log documenting how data was stored, backed up, and anonymized to ensure participant confidentiality and data security.

3. Data Analysis

Coding Process: (Date) Documentation of the coding process, including:
- Initial Coding: Initial codes developed, their definitions, and examples from the data.
- Codebook Development: Evolution of the codebook over time, with notes on how codes were refined, merged, or dropped.
- Analytical Memos: Memos capturing the researcher’s thoughts, insights, and emerging interpretations during the coding process. These memos can document the rationale for coding decisions and the development of theoretical connections.
- Diagrams or Visual Representations: Visual representations of the coding structure, relationships between codes, or emerging themes can be included to enhance understanding of the analytical process.
Theme Development: (Date) Documentation of how themes were identified, developed, and refined, including:
- Theme Definitions: Clear definitions and descriptions of each theme, with illustrative quotes from the data.
- Theme Relationships: A description of how themes relate to one another, potentially using diagrams or visual models.
- Negative Case Analysis: Documentation of how data that did not fit the emerging themes was considered and addressed. This might involve refining themes, developing sub-themes, or explaining why certain data points were considered outliers.
Software Log: If qualitative data analysis software was used, a log documenting the software program, version number, and any specific features used for analysis (e.g., query functions, coding tools, or visualization options) should be included.

4. Trustworthiness Strategies

Member Checking: (Date) Documentation of member checking procedures, if used. This might include:
- Method: The method of member checking used (e.g., returning transcripts, member check interviews).
- Participant Feedback: Summaries of participant feedback received, including any modifications made to interpretations based on their input.
- Ethical Considerations: Documentation of how ethical considerations related to member checking were addressed, such as ensuring participant anonymity and addressing potential emotional impacts of revisiting sensitive data.
Peer Debriefing: (Date) Notes from discussions with colleagues or experts regarding the study’s methods, analysis, and interpretations. This includes documenting diverse perspectives and any adjustments made based on peer feedback.
Reflexivity: Ongoing reflections on the researcher’s positionality, potential biases, and how these might have influenced the study. This can be documented in the field journal or in separate reflexivity memos.
Audit Trail Review: If an external auditor is involved, documentation of the auditor’s role, feedback, and any modifications made to the research process based on their recommendations should be included.

5. Dissemination

Manuscript Drafts: (Date) Different versions of the manuscript, showing how the findings were written and revised.
Presentation Materials: (Date) Slides, posters, or other materials used to present the study’s findings at conferences or meetings.
Data Archiving Plan: A plan for how and where research data will be archived for long-term preservation and potential future use.

Best Practices

Best practices for maintaining an effective audit trail in qualitative research center on ensuring transparency, organization, and accessibility.

The goal is to create a comprehensive and easily traceable record of the research process, enabling an external reviewer to follow the study’s path and assess the trustworthiness of the findings.

Remember, the audit trail is an ongoing process that should be maintained throughout the study, not just a task completed at the end.

1. Maintain Chronological Order

A chronological audit trail helps demonstrate the evolution of the research and provides a clear record of decisions made at each stage.

This can be achieved by:

Dating all entries and documents: Meticulously dating every piece of information added to the audit trail ensures a clear temporal order of events and decisions.
Creating a timeline of research activities: A visual timeline can be beneficial in representing the overall flow of the research, highlighting key milestones and changes made along the way.
Recording the sequence of analytical decisions: Documenting the sequence of analytical decisions showcases the progression of data analysis, revealing how codes, themes, and interpretations emerged.

2. Use Clear Organization

A well-organized audit trail is crucial for easy retrieval and review of information.

Consider the following:

Developing a consistent filing system: Implement a logical system for organizing files, ensuring all data, analysis products, and process notes are stored in a structured manner. For instance, separate folders could be used for raw data, coded data, memos, and meeting minutes.
Creating clear labeling conventions: Using consistent and descriptive labels for files, codes, and memos makes it easier to locate specific information within the audit trail.
Maintaining separate sections for different types of data: Separating different types of data, like interview transcripts, field notes, and documents, allows for efficient access and comparison.

3. Include Detailed Documentation

Detailed documentation is essential for providing context and understanding the rationale behind decisions made during the research process.

Focus on:

Recording the rationale for decisions: Documenting the reasoning behind methodological choices, sampling strategies, and analytical decisions enhances the transparency of the research process. This includes justifications for any changes made to the initial research plan. For example, if a researcher decides to collect additional data, the reasons for this decision should be clearly documented in the audit trail.
Documenting changes to research design: Any modifications to the research design, such as changes in data collection methods or sampling procedures, should be thoroughly documented, along with the reasons for these changes.
Noting challenges and solutions: Recording the challenges faced during the research, such as difficulties in recruiting participants or unexpected events impacting data collection, and how these were addressed, adds to the authenticity and trustworthiness of the research.
Keeping meeting minutes with advisors/team members: Maintaining records of discussions and decisions made during meetings with advisors, collaborators, or research team members contributes to a comprehensive account of the research process. This includes documenting any disagreements or alternative perspectives considered.

4. Ensure Accessibility

An accessible audit trail is one that can be readily accessed and understood by others.

Implement these practices:

Make files easily retrievable: Organize the audit trail in a manner that allows for easy retrieval of specific documents and information. This might involve using a digital filing system with clear search capabilities.
Create backup copies: Regularly backing up the audit trail ensures that valuable data and documentation are not lost.
Use consistent naming conventions: Consistent naming conventions for files and folders facilitate quick and efficient location of information.
Consider digital organization tools: Digital tools like qualitative data analysis software can assist in organizing and managing the audit trail. Some programs offer features for storing raw data, codes, memos, and annotations, creating a centralized and searchable repository for the audit trail.

external reviewer

Lincoln and Guba (1985) recommend an independent auditor to review the audit trail in qualitative research for evaluating dependability and confirmability.

The auditor’s role is similar to a financial auditor who examines a company’s accounting methods and verifies the accuracy of its financial records.

In qualitative research, the auditor scrutinizes the process and product of the study.

Core Audit Components

Dependability Audit: The auditor examines the methods used for data collection and analysis, ensuring they align with accepted practices and are documented clearly enough to be replicated. This audit focuses on the stability of the data and whether similar results would be obtained if the study were repeated.
Confirmability Audit: The auditor verifies that data supports the interpretations and conclusions made by the researcher. This audit aims to establish that the findings are grounded in the data and not influenced by the researcher’s biases.

The external auditor’s feedback can enhance the trustworthiness of the research and ensure the rigor of the study’s methods.

The auditor can examine the documentation to ensure completeness and consistency, and offer feedback on the transparency and trustworthiness of the research process.

Ideally, the auditor should be involved from the start of the project, allowing them to guide the creation of the audit trail and provide ongoing feedback.

However, this continuous involvement needs to be balanced with maintaining their objectivity.

Technical support and documentation

The use of computer software in qualitative data analysis can facilitate the creation and management of an audit trail.

Some software programs have built-in features for tracking codes, memos, and decisions made during analysis, which supports the documentation process required for effective auditing.

Key Benefits

Enhancing dependability and confirmability: The auditor’s independent perspective helps ensure the stability of the data and confirms that the interpretations are rooted in the data.
Keeping the researcher ‘honest’: The auditor’s scrutiny encourages the researcher to be thorough, transparent, and accountable in their research process.
Providing a thorough check on researcher bias: The auditor’s objective review helps mitigate potential researcher bias and enhances the neutrality of the findings.

Potential Challenges

Potential for co-option: The auditor’s prolonged involvement might lead to them becoming too invested in the study and losing objectivity.
Availability of qualified auditors: Finding an individual with the necessary expertise and time commitment to conduct a thorough audit can be challenging.
Cost implications: Hiring an external auditor can add to the research expenses, especially for smaller-scale projects.

Reading List

Halpern, ES. (1983). Auditing Naturalistic Inquiries: The Development and Application of a Model. Unpublished Doctoral Dissertation, Indiana University.
Lincoln, YS. & Guba, EG. (1985). Naturalistic Inquiry. Newbury Park, CA: Sage Publications.
Malterud, K. (2001). “Qualitative research: Standards, challenges and guidelines.” The Lancet. 358: pp. 483-488.
Schwandt, TA. Halpern, ES. (1988). Linking Auditing and Metaevaluation: Enhancing Quality in Applied Research. Newbury Park, CA: Sage Publications.

Audit Trail In Qualitative Research