Some Notes on Analysis
Comparing Cohorts of Learners
This is a topic that emerged in the network recently so might be a good starting point for discussion. In educational inquiries, and particularly, small scale, qualitative projects comparing cohorts with one another can happen, but is not often done. Here are a couple of reasons why:
- SoTL often explores teaching interventions, which are designed with the aim to improve student learning experience and/or grade marks, so enabling one group of learners to experience this and another group not to, is not an ethical approach to a SoTL project. (There are ways to address this, with cohort swapping etc if you have your mind set on comparing).
- Qualitative research does not rely on cohort comparison for reliability, this can be achieved by other means, this blog post (link opens in new tab) is a good reflection on some of the issues, and leads to more complex discussion about reliability, validity, and generalisability and the purpose of (small scale) qualitative studies: Another way of thinking about the quality of your educational inquiry might be verification:
Morse, J. M., Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2002). Verification Strategies for Establishing Reliability and Validity in Qualitative Research. International Journal of Qualitative Methods, 13–22. https://doi.org/10.1177/160940690200100202
Tracy (2010) developed an approach to quality in qualitative research that can easily be applied to SoTL projects: Tracy, S. J. (2010) ‘Qualitative Quality: Eight “Big-Tent” Criteria for Excellent Qualitative Research‘, Qualitative Inquiry, 16(10), pp. 837-851.
A good overview of different types of reliability and validity is in this blog post by research collective.
Back to simpler things
Thematic Analysis
What is in a name? Here we are literally looking for themes. One way to think about Thematic Analysis (TA) is that we are looking for patterns. When we read open text comments in questionnaires, or analyse interviews–we are trying to identify topics (themes), this is usually done by highlighting words, phrases, sentences of similar meaning (coding). This is an iterative process and comes with a significant amount of widdling things down (before reshuffling them into larger categories again). So that was the ‘technical explanation’–a fantastic paper discussing the context, different types of TA and how to do it is the following reference:
Braun, V., Clarke, V., Hayfield, N., & Terry, G. (2019). Thematic Analysis. In P. Liamputtong (Ed.), Handbook ofResearch Methods in Health Social Sciences (pp. 844–860). Springer Nature Singapore Pte Ltd. https://doi.org/10.1007/978-981-10-5251-4_103
Some authors speak of emerging themes. However, you might want to consider and reflect on the following questions:
- Can themes or codes truly emerge or are you constructing them?
- If you think they emerge: How do they do this?
- If you construct these, can you find a critical friend, or colleague, and will they construct the same codes/themes?
- Reflect on differences and overlaps
- Transparency of process is important
#UofGSoTL 