Issues with methodology (social science)

C

My methodology chapter is a mess. I did mixed methods. but poorly. (I wrote a survey administered online to pupils and then did interviews with teachers in a second phase of research). There were no pilot studies or any other test retest type occurrences. I didn't follow a plan; I sort of just went off and did research and am now paying the price. I have no idea how to write up issues of reliability and validity as there I did neither in preparation for the research. I did no background work or checking, so how do I write this up academically? Can I? Am I doomed for the viva? I'm submitting in 2 months and am working around the clock trying to finish everything and feel like it's pretty hopeless right now. Any advice will be great!

Z

I do not think you are doomed and I think you can pull this off- the beauty of social science research is that it is very reflective. If you wish you could change something in your approach, you can reflect on that in your thesis. This may be an approach you need to take when discussing reliability. Validity however can easily be discussed depending on what paradigm you are approaching your research from. If it is interpretivist, then this will be easy.

I did no pilot studies for my research as I was working with educational institutions and gaining access was challenging enough, never mind trying to gain access for a pilot study on top of recruiting for the main study. Time was tight too. I am wondering if you had any of these issues? Did you opt for an online survey for convenience? It could also be seen as more ethical as an online survey will minimise the time demands of the pupil participants- it does not disrupt their daily schedules nor does it disadvantage the scheduling of the school e.g. they are not sacrificing learning time to complete your survey. Consider access and ethical issues like these to frame the approach you took in a positive light, but then also leave room to reflect on why a pilot would have been useful.

I hope this helps :)

C

Thanks Zutterfly. I really didn't and don't think paradigm-wise, despite using one. (It's currently interpretivist but I'm not really thinking it makes sense. I've done some reading on pragmatic worldviews, but am unsure I can change this section in detail with 2 months to go). I just don't really understand how it really applies since I had to determine all of this about 2 years after collecting the data. My main concern is that there is no way to determine reliability or validity in either my qual or quan methods and I still have to explain how I did so, even though I didn't. That's my main concern right now.

B

Have you been specifically told that you had to obtain reliability and validity? It might be that you have to run these tests if they are needed in the method section.

See what your supervisor suggests, Methods chapters in Social Sciences can be so difficult to write, having to justify methods and analytical approaches, as well as reading the epistemology behind them. So see what is required from you and what your potential examiners might want to see.

Avatar for Pjlu

Hi there, just wondering with your qualitative results-you mention that you didn't do any member checking (I think?). However, you must have used a method for analysing or interpreting your qualitative data. So for example, you would have coded for categories or sub themes and then used inductive methods or similar for clustering these into themes which you then would have described and interpreted. What I am trying to say I think is that the methodology you used to interpret your qualitative data provides validity and reliability.

A lot of these methods can be described as variations on inductive thematic analysis-which is a valid interpretivist methodology. Likewise with your questionnaire (for students), in analysing the quantitative data, you must have used simple descriptive statistics-which are valid provided you outline your use of these and clarify how far you are taking the results. I agree with Zutterfly, getting results from schools and institutions can be a challenge in itself and pilot studies are not always possible. When you discuss your results, comparing the findings from both data sets, then using the main themes and findings, cross analysed against the literature should be okay-provided, as mentioned by Zutterfly-you outline your understanding of the limitations.

C

Quote From butterfly20:
Have you been specifically told that you had to obtain reliability and validity? It might be that you have to run these tests if they are needed in the method section.



Yes, I was told it needs to be a part of it, but of course with no direction or assistance whatsoever.

Quote From Pjlu:
Hi there, just wondering with your qualitative results-you mention that you didn't do any member checking (I think?). However, you must have used a method for analysing or interpreting your qualitative data. So for example, you would have coded for categories or sub themes and then used inductive methods or similar for clustering these into themes which you then would have described and interpreted.


No, there was no member checking done. I didn't know it was concept until very recently. (My fault I didn't even look at a textbook when I was doing all of this!) I used thematic analysis (I guess). I basically just read and re-read over the transcripts until I found themes or what I thought were themes and kept changing them until my supervisors agreed.

A lot of these methods can be described as variations on inductive thematic analysis-which is a valid interpretivist methodology. Likewise with your questionnaire (for students), in analysing the quantitative data, you must have used simple descriptive statistics-which are valid provided you outline your use of these and clarify how far you are taking the results. I agree with Zutterfly, getting results from schools and institutions can be a challenge in itself and pilot studies are not always possible. When you discuss your results, comparing the findings from both data sets, then using the main themes and findings, cross analysed against the literature should be okay-provided, as mentioned by Zutterfly-you outline your understanding of the limitations.


I only use basic descriptive statistics to obtain percentages of how many pupils experienced what. I shall do the best I can with this. Thanks!

Avatar for Pjlu

Cindylouwho, did you discuss any theorists when you outlined your epistemology for Mixed Methods and interpretivist studies. I found that Mixed Methods just seemed to fall into pragmatism and most of my (brief) research into this turned up practical texts like Creswell (Education) 2005 & 2013 (Sage) which mainly outline 'how to' not 'why you should use it'.

However when you discussed interpretivsm you would have discussed a theorist when mentioning your epistemology (how you know). So for example, who were the theorists behind your use of interpretivist theory-and if you are not sure, grab a copy of Creswell's Qualitative Inquiry and Research Design: Choosing Among Five Approaches (Sage-2013) and go to the chapter that provides a table of the five approaches.

One of them will be somewhat like your method for processing your qualitative data and this can be used to help establish validity of method. Braun and Clarke (2006) are great on Thematic Analysis-just don't be frightened off by some of their insistence on NOT using the terms 'emerging from the data'-they hate the word 'emerge' and their reasons are good, but sometimes you just have to use the word to get your message across.

47476