Teaching Research Methods: To embed or not to embed...the rise of a third way?

Helen Williams, University of Nottingham

In recent years, discussions around whether to embed research methods training in substantive modules or to teach them in standalone modules have received a great deal of attention in the UK. Recent research by Malcolm Williams et al. (2016) indicates the falseness of this dichotomy, showing that students who learned statistics through substantive modules do not reach the deeper level of understanding achieved in standalone modules. Thus far, the discussion seems to have been structured as an either/or choice: either students take a substantive module with a sprinkling of methods hidden in the content, or they take a 'methods module' that students frequently feel lacks clear links to the rest of their degree. This case study reflects on initial results from an attempt at a third way, consisting of a methods module that uses a substantive theme from the rest of the students’ course. This module concerned also approaches the employability aspect of methods training more explicitly by employing Excel rather than a specialist statistics programme. This case study analyses some of the successes and failures identified in the first iteration of taking this approach.

Approaching the problem

Following on from the first year, first semester core module Introduction to Comparative Politics, I used post-materialism as the core theme for the students’ introduction to research methods. Contrary to the previous research methods module, Designing Political Research, which explicitly labelled itself as a research methods module, this module’s name — Culture and Values in a Changing World — highlighted the substantive theme. It focused on topics with high levels of saliency and student interest, e.g. the Clash of Civilisations thesis; xenophobia, nationalism, and anti-immigrant voting; and explaining Russia. Each week’s content combined an element of research methods, a substantive theme, and a statistical technique. The core readings exhibited at least two of these three characteristics; most readings combined all three.

Because of limitations to contact hours and staffing, the module was delivered using a blended learning approach. Students had a weekly lecture and seminar, both of which were delivered face-to-face. The lecture introduced the week’s theme and gave an overview of research findings in the area, concentrating on analysing these in the context of the week’s research element, e.g. crafting a research question, developing a testable hypothesis. Seminars were a combination of discussion of the weekly reading and activities to get the students working towards their final assessment: an original data report.

Students additionally had to watch a software demonstration video teaching the week’s statistical technique, then complete a weekly quiz that required them to manipulate the dataset and compare their results to the weekly reading or lecture materials in order to find the correct result. Students were allowed two attempts at each quiz without a time limit beyond the weekly deadline; the mark obtained was the average of the quiz results in order to discourage random clicking on the first attempt.

What went well?

Without explicitly labelling it as such, the students were doing near-weekly data replication exercises, comparing more recent results to the results they were reading. The focus on interpretation seemed to work very well. For the first time in nearly a decade of teaching research methods, not a single student questioned why they had to do research methods and what learning statistics had to do with their politics and IR degree. From discussions with the students, this arose for two reasons:

  1. Using Excel made it obvious to students how they were learning a directly employable skill. When questioned, students highlighted the need to increase their familiarity with Microsoft Office programmes because of their ubiquity in the workplace.
  2. The content was so thoroughly embedded in the big questions of politics and IR that they didn’t feel that it was something separate to what they were learning on the rest of their degree. Many students engaged intensely with the content, going well beyond the expectations of the module in their enthusiasm to explore their chosen topic.

The success of this module is also demonstrable in the fact that fully half of the cohort (130 students) have voluntarily enrolled to take an SPSS-based statistics module as an option in the second year of their degree. The important point here is that this is not a Q-step effect: only around 10% of the students electing to take further statistics training in their second year are doing so as part of the Q-step curriculum, and neither of these modules are part of the Q-step core requirements.

Lessons learned

The structure and mode of delivery of this module placed a significant burden on the module convenor. Just identifying ideal weekly readings that combined a particular aspect of research design, a particular statistical technique, and one of the substantive themes took more than a week of staff time. It took around 10 hours to create each software demonstration and accompanying quiz. And committing to using post-materialism as the core theme, with the accompanying World Values Survey and European Values Study, was an act of unresearched naivety that became apparent when I realised that Excel versions of these surveys did not exist and therefore had to be created from scratch.

Most of the other issues highlighted by students in the module evaluations had more to do with teaching style and inevitable pitfalls from the first time of running the module. Therefore, I hope that many of these complaints will disappear in the second or third time of running. Despite these issues, the results far surpassed expectations and provide clear evidence in support of taking a third way approach (Figure 1). The success of such an approach depends on very close working with colleagues and a supportive Head of Department/School to enable experimentation.

Figure 1. Module evaluation results

Module Evaluation Results

* Net agree=(strongly agree + agree) – (disagree + disagree strongly)