Understanding quantitative research articles

1 Understanding research based articles

When reading a research article, ask the following questions to help you understand the structure and argumentation of the article. If you still cannot understand it, this may be because (1) the author has not communicated his/her ideas clearly, or (2) you simply need more background knowledge of the topic to understand it - not your fault if you're new to the field.

1. What is/are the main objective(s) of the study?
2. What previous work motivated the study?
3. Does the author state a specific hypothesis or set of hypotheses? If so, what is it / what are they?
4. If a specific hypothesis is not stated, does the author pose a clearly defined research question or questions? If so, what is it / what are they?
5. How does the author test the hypothesis or address the research question? In doing so, does the author identify the following? In your paper, you can usually identify or summarize these research methods in 1-2 or a few sentences, and focus more on the results or findings.
   1. subjects
   2. materials or stimuli used
   3. procedure for collecting the data
   4. methods of data analysis
   5. results
6. Do the results convincingly support the author’s hypothesis? Or do the results adequately address the research question? Explain why or why not.
7. In the author's opinion, what are the implications of this study? Do the author’s conclusions here seem valid or supported by the data?
8. In your opinion, what are the implications of this study?
9. In your opinion, what are the strengths and weaknesses of this study?
10. Does this article make a good contribution to our/your understanding of the topic?

2 Understanding statistics and their usage in a research article

You may read papers that involve discussion of experiments, statistics and statistical data, including particular statistical procedures (e.g., ANOVAs, chi-square, regression, correlation, t-tests, factor analysis, Cronbach's alpha, and many others). For now, you can skip over all the numerical data and statistical terminology, and focus on the following

2.1 Look for the big picture

It’s important not to get too caught up with individual data points or a tiny section in a really big dataset. Focus on the big picture – the overall purpose of the study, what the researchers were trying to do, whether the data support their hypothesis, and what it means for you as a teacher or consumer of research.

2.2 No agendas

This should go without saying, but approach data as objectively as possible. Researchers usually have a hunch about what they are looking for, but they should not let your preconceived ideas influence the results. Likewise, you should have an open mind as well when evaluating others’ research. If you go to length looking for some specific pattern, you're probably going to find it. It'll just be at the sacrifice of accurate results.

2.3 Look outside the data

Three things are important: Context, context, context. The context in which the researchers did their study can affect the results or validity of their findings. Sometimes this will be in the form of other research on the topic, other data, or other studies. The more you know about how the data was collected, where it came from, when it happened, and what was going on at the time, the more informative your results and the more confident you can be about your findings. For example, would the environment in which they did their research affect its validity – would it cause doubts about whether their findings would be valid in other contexts? Would there be problems in applying their findings to your teaching context or research context? What differences between your context and theirs might cause problems for generalizing or applying the findings to your situation?

2.4 Ask why

Finally, and this is the most important thing I've learned, always ask why. If a researcher reports some correlation, you should think about whether it makes any sense. If it does make sense, then cool, but if not, dig deeper. Numbers are great, but you have to remember that when humans are involved, errors are always a possibility.

See also: Understanding statistics in research articles