Challenges of Embedding & Maintaining Substantive Interests in QM Courses

Challenges of Embedding & Maintaining Substantive Interests in QM Courses

Brian Fogarty, University of Glasgow

One of the most difficult courses to teach in undergraduate politics degree programmes is the Political Analysis course (or Quantitative Methods or whatever name the university uses). Amongst numerous challenges is the challenge of engaging politics students in the course material. The overwhelming majority of social and political science students did not do well in maths and related courses, and the sight of Greek letters and R/Stata/SPSS interfaces typically produces anxiety and illness. Common questions and complaints include ‘Why do I have to take this course?’, ‘How does this relate to what I am studying in other courses?’, ‘I hate R/Stata/SPSS’, etc. As the instructor, you hope for student evaluations along the lines of ‘the instructor made the course as engaging as possible given the material.’

Below, I discuss observations and best practices for engaging undergraduate politics students in QM courses by embedding and maintaining substantive interests into the analysis. This is based on nearly 10 years of teaching undergrad QM politics courses and 2 years of teaching in Glasgow’s Q-Step programme. While Q-Step students are self-selecting in and typically the politics QM course is compulsory, similar challenges arise. Obviously, everyone has their own preferences, techniques, and goals in teaching QM, but this is simply what has worked for me.

Emphasize the Relevance of QM. . . All the Time

Tackling the most common complaint of ‘why do I have to take this course?’ requires in the first lecture and then every following lecture an emphasis on the relevance of QM. To put it simply, ‘how do we know what we know?’ Students are learning all sorts of substance in their courses, but they likely never think ‘how do we know this?’ This is the perfect opportunity to sell QM to skeptical students.

Ask Questions about “Obvious” Statements

It is not enough to be vague or general on this. You need to provide explicit examples. Some examples I’ve used include:

1. How do we know that the UK is more democratic than Russia?
2. How do we know whether one country is more unequal than another? 3. Are the political media biased?
4. Why did party X win the election?

The point of such questions is to push students to think systematically, analytically, and to start thinking about how we measure concepts. For example, asking students to list facets of democracies and how one would might measure each element begins the process of thinking empirically and quantitatively.

Related is the ubiquitous goal of QM courses making students ‘better consumers of quantitative information’. Before university, most politics students likely would be able to provide responses to the above questions based on what they’ve heard in the media, from friends, family, etc. In asking questions like the above you will probably get a response that is informed by such sources. This is the point where we challenge the information to hold up to quantitative scrutiny and hopefully plant in students the ‘how do we know this?’ desire.

Bring in the News

There are a variety of ways this can be done. One can bring in headlines that are clearly misleading. Or, show the difference between data-informed news and data-lacking news. One obvious attention grabber is bringing data visualisations from the news; such as immigration patterns from the middle east to Europe. However, undergrad politics students are typically not going to be creating fancy data viz.

Instead, bring in current news that links directly to the QM methods you are teaching. This is sometimes difficult to do in politics depending on what is happening. Also, you do not want to spend too much time on hunting down stories all the time.

Obviously election seasons provide numerous opportunities. The stories you show students might be based on what you know about some topic. Increasingly due to the Upshot and Monkey Cage, one is able to contrast news on a topic within the same news outlet. For example, show a story on voter fraud that has little to no quantitative information and then show a story that is rife with it. Not only does this get students to start thinking quantitatively, but it can also show students what learning QM allows them to do.

In lieu of good politics news articles, you can always find medical headlines that are mis- leading even in major papers like the New York Times. The usual points one can raise are correlations, control variables, spurious relationships, observational vs. experimental studies, and multivariate analysis. These are QM techniques that many students will know by the end of their course. While these are not substantively political topics, you can emphasize the potential policy implications. Some real headlines I have used include:

1. Bottled Water Linked to Healthier Babies
2. Drinking Beer Aids in Marathon Recovery
3. Watching an Hour of TV Reduces Life Span by 22 Minutes 4. Obesity Rate for Young Children Drops by 48% in a Decade

Issues with the first 3 are quite obvious, but the 4th one requires a bit more information. The New York Times article notes that obesity rates in 2-5 year olds in the US dropped from 14% in 2004 to around 8% in 2012. Technically that is a decrease of around 48%, but most people are going to read that as a drop by 48% in real numbers (e.g., from 50% of children obese to 2% of children obese). This example is not technically QM, but it does reinforce to students to be smart about numbers being used in the news.

Bring in Popular Books that Use QM

To break up the monotony of doing statistics in QM courses, I have students read and discuss popular books that use QM. This can be a bit tricky because hitting the right level can be difficult.

The quintessential book for this is the original Freakonomics - less so the follow-ups. Freakonomics uses compelling narrative to draw students into thinking about how to use QM and develop research designs to answer often quirky questions. It is an easy read for undergrads and those who want more technical information can find citations to the original academic sources. Using Freakonomics led to one of my favourite QM teaching moments:

When talking about the chapter on how increasing abortion rates has led to a decrease in crime rates in the US, one very socially conservative student stood up out of his chair, angered, and said “there is no way you can empirically show that is true”. Another student immediately responded with “that is exactly what they show”. And I said “Exactly!”

I have also used Malcolm Gladwell’s books to show what else can be done with QM, but also to point to his (frequent) logical leaps that are not supported by empirical evidence. I once used Nate Silver’s The Signal and the Noise but my students found it too dense.

Use Examples from the Geographic Area of the University

Another technique I frequently use to embed substance into QM courses is to use data examples from the geographical area of the university. This is particularly effective for students who grew up in the area.

One example I used when living in St. Louis, MO (USA) concerned descriptive statistics for the racial composition of the city:

The mean percentage white in voting precincts is roughly 48% and the median is around 47%. And so the average voting precinct is roughly half white and roughly half black (the small percentages of other races are excluded for this example). The values for the mean and median make St. Louis appear like a very mixed and non-segregated city. However, the standard deviation for percentage of whites in voting precincts is 42%. That seems strange, doesn’t it? Let’s look at a simple histogram to see how the mean and standard deviation work together to give us insight into this variable.

Figure 1:

We can see that the distribution of percent white in voting precincts is actually U-shaped. Looking closer we see that in roughly 35% of precincts whites make up 0 to 6% of the population, while in roughly 20% of precincts whites make up 94 to 100% of the population. Therefore, the majority of voting precincts in St. Louis are either almost all black or almost all white. This explains why the value for the standard deviation is so large. If we just relied on the mean, we would have completed missed that St. Louis is noticeably segregated by race; and in fact one of the most segregated cities in the US.

Students found that providing empirical and quantitative evidence for local things they intuitively knew was one of the most compelling reasons to study QM. In fact, several students cited this example for influencing them to pursue graduate study in urban public policy where they could continue to use QM.

Data

Finally, the data you use in the course and for students’ assignments can help (or hinder) efforts of embedding substance into QM courses. Below are several observations and suggestions:

Use Real Data

One common complaint I have heard from students who have taken other QM courses (possibly in different subjects) is their general dislike of use fake and/or ‘toy’ datasets. For instructors, using fake or toy data can make teaching data analysis much simpler as the general properties are known and there are not too many moving parts. However, such datasets turn off students trying to learn QM because it removes the substance and interest in using data to study political questions. So, instead of students finding QM engaging for their study of politics, they instead find QM courses ‘dumb’, ‘stupid’, ‘worthless’, etc.

Of course, you cannot just say to students to go download raw data files from the UK Data Service or similar sites. You still need to clean, prepare, and annotate data such as the British Election Study to make things easier for students. And you will likely cut away variables that you find irrelevant or not interesting. Just resist the urge to slash it down to just a handful of variables. Students need to get used to the vastness of social science data and having students analyse large datasets may spark interest in further QM study.

Limit Choices for Data Analysis Assignments

If your course contains a research project where students need to analyse data and write-up a research paper, only let students pick data from a restricted set. Try to make the restricted set as broad as possible by including datasets that appeal to the various subfields. You do not want to limit students interests, but in QM courses there needs to be a focus on the feasibility of data projects. In the long-run using a restricted set of possible datasets benefits both students and yourself as the instructor.

I learned this lesson the first time teaching QM in politics. I allowed students to find and use any data they wanted to for their final research project. What I thought of as providing freedom to students turned into an absolute nightmare for myself and the students. Everything that could go wrong did, massively increasing my workload for sorting out datasets and making students’ anxiety shoot through the roof. After that first class, my thought on data choices was ‘keep it simple, stupid’.

Can’t Satisfy Everyone

Even if you are teaching QM to just politics undergrads, you will likely not satisfy all your students’ data interests; and you should not worry too much about this. I provide a set of datasets that roughly cover the various main subfields in politics. You can always add in additional datasets as long as you have a good sense of their composition.

Satisfying all interests became a significant issue when I first taught the Linear Regression course (the second course in a five course sequence) in Glasgow’s Q-Step programme. Our QM programme provides students from politics, sociology, public and social policy, economic and social history, and central and eastern European studies a ‘with Quantitative Methods’ degree; essentially a minor in the US equivalent. The first year cohort were mostly sociology students and I quickly realised that I would need more sociology examples. The real issue concerned their main written assessment where students had to replicate a linear regression analysis from a published article. Looking for an article from this century that used plain OLS was difficult, and I learned two key things: first, there are no sociology journals that have open data and code; second, political science is way further ahead in the open data and replication movement than most other social sciences. To head off the inevitable complaints, I told the students numerous times of the problems in locating sociology open data. So, I wound up using a political science article that analysed US election data. The verdict was that the students hated it because it was politics and too American. Thus, I am now continually hunting for open data across the social sciences, but often to minimal effect.