Understanding statistics in research articles

1 A few important numbers

In regard to the actual numbers, a few are important and worth looking at in the statistical results; we will discuss these more later.

In statistical many tests (ANOVA, MANOVA, t-tests, chi-square, logistic regression, HLM, etc.), the one number that is important is the p value, referring to probability. In simple terms, one could say that this means the probability that the statistical results are not by chance or not a fluke, in other words, the likelihood that the findings of an effect of X on Y is a strong probability and likely valid, or that the findings of a difference between X and Y are valid. We usually look for a p-value of p≤.05. If it is at .05 or less, then the findings are probably valid statistically.

Some tests look for the degree of a relationship between X and Y, that is, a correlation (or regression analysis). You will see an r- value or a ρ (Greek letter rho) value, indicating the strength of the correlation or relationship between X and Y. Usually, a correlation of r=0.6 or above is rather strong one in social science research; a correlation of 0.4-0.6 is fairly strong, and even in the 0.2-0.4 range is moderate. Some effects under 0.2 could be statistically valid. For correlations, one has to consider whether the relationship makes sense or is meaningful. For example, one might find a statistically valid correlation between the amount of pizza that people eat and their scores on the TOEFL, but it would not be meaningful. One must also remember that correlation does not necessarily mean causation. If X correlates with Y, that does not mean that X causes Y – unless one has other good reasons to believe that X would naturally be a cause of Y.

Occasionally, you may see a Cronbach's α (alpha), e.g., to measure how accurately and consistently two raters scored a test, or how consistently students answered a test item. A level of α=.8 or above would be considered a rather strong correlation or level of consistency; .6-.8 is moderately strong, even somewhat below α=.6 might be good in some cases.

2 Hint for reading experimental papers

Often for your purposes at present, it is not necessary to read all the statistical details. Often you can skip those sections. Such papers often contain a main section devoted to explaining the experiment and the results. At the end of each experimental section of the paper, there might be a brief subsection called 'Discussion', which summarizes the results in less technical or mathematical language. You can focus on this, as well as the introduction to the experimental section to get the main idea of what they did and what they found. Then such papers often have a main section (often called ‘General discussion’) after that, in which they discuss in more general terms the results, what they mean, how they prove or don’t prove their hypothesis, the significance of their results, and their implications. You should focus especially on the main discussion or general discussion section of the paper, and the authors’ further discussion of the significance and implications of their findings. You can then respond to and critique what they say here.

Also, any kind of study, especially in a journal article, might be somewhat broad in scope, or lengthy. In that case, simply focus on the aspects that are relevant to the topic of your paper. Do not try to summarize or critique the whole paper in that case – just focus on the aspects that are important for your own paper.

See also: Understanding quantitative research articles