The Impact of a College Skills Course

Abstract

Salt Lake Community College offers a college skills course, "Essentials of College Study" (hereafter referenced as CS). Do students who take this course have higher GPAs and graduate at higher rates compared to students who don't? This question was addressed by SLCC's Institutional Research office in Gold Paper 3 ("Measuring the Effectiveness of the Developmental Education Course LE 1020," April 3, 2012). This paper caused a stir at the college, and, as a colleague noted in email correspondence, "there was a great deal of talk about requiring that all incoming students be encouraged or mandated to take the course."

This article reviews the literature on the impact of college skills courses, summarizes the findings of Gold Paper 3, raises some concerns about the methodology used in that study, and presents the findings of a re-analysis of the impact of CS that uses a different methodology on a larger dataset.

Summary of Results

1. Gold Paper 3 reported that CS was associated with improved student performance. However, that finding did not hold up in our re-analysis and was likely an artifact of the methodology used in the original study.
2. In our re-analysis we matched CS and non-CS students on a range of performance and demographic variables, interpreted the treatment (taking CS) according to the "intention-to-treat" paradigm familiar from randomized clinical trials in medicine, and then compared the two groups statistically. Results: CS had no association with final GPA or graduation at the two, four, or six year year marks; term-specific GPA in the semester of, or in the semester following, the course; or retention in the semester following the course.
3. In an additional analysis, we selected five courses to compare with LE 1020 and EDU 1020: COMM 1010 (Elements of Effective Communication), HLTH 1020 (Foundations of Nutrition), FIN 1050 (Personal Finance), PSY 1010 (General Psychology), SOC 1010 (Intro to Sociology). We found that among the students who took these seven courses in the first semester, those who took EDU 1020 or LE 1020 tended to have lower GPAs and lower graduation rates than students who took these other courses.
4. These findings should not be understood as a criticism of the course. Indeed, it may be unrealistic to expect that a single course--out of the many that students take during a period of very rapid change in their development--should impact performance measured years later (in the case of final GPA or graduation). An assessment of the course based on students' reported experience, or focusing on different outcomes, such as self-efficacy or attitudes towards life-long learning, may be more appropriate.

Figure 1. A comparison of average final GPAs associated with courses students took in their first term at SLCC using a per-protocol approach (left) and an intent-to-treat approach (right). The per-protocol approach considers only students who passed the course; the intent-to-treat approach considers all students who enrolled in the course. The distribution of average final GPA differences across these courses is similar in the two panels: the CS courses, in gold, tended to have lower GPAs. But the overall level with respect to the institutional average was extremely sensitive to the choice between PP and ITT.

Figure 4. A comparison of six year graduation rates associated with courses students took in their first term at SLCC using a per-protocol approach (left) and an intent-to-treat approach (right). The per-protocol approach considers only students who passed the course; the intent-to-treat approach considers all students who enrolled in the course. The distribution of four year graduation rate differences across these courses is similar in the two panels: the CS courses, in gold, tended to have lower rates. (Note that EDU 1020 does not have a six year graduation rate because it has not been in existence long enough.) But the overall level with respect to the institutional average was sensitive to the choice between PP and ITT.

Research Report

(requires College login)