In my recent commentary on the Brookings article about why more adults weren’t finishing community college, I noted that the authors had published a more extensive research paper in the Educational Evaluation and Policy Analysis Journal.
I was able to obtain a copy of the journal article. It provided much more detail about the research and the process of eliminating certain students from the Virginia Community College System (VCCS) cohorts analyzed. Given the first time nature of a study like this, I decided to review the article in more detail.
During the 2009-2014 period, nearly 200,000 students attending the 23 VCCS colleges met the researcher’s screening criteria of students who earned at least some college credits, left VCCS without earning a degree or credential, and did not transfer or reenroll for at least three years after dropping out. Records obtained for these students include detailed information about each term in which the student was enrolled including their program of study, courses taken, grades earned, credits accumulated, and financial aid received. The records also include demographic information such as gender, race, and home zip code. The researchers were able to access National Student Clearinghouse (NSC) records to track student reenrollment at another institution as well as Virginia Employment Commission (VEC) records tracking quarterly earnings for students for up to five years prior to VCCS enrollment and indefinitely during and after VCCS enrollment.
Nearly two million students enrolled at VCCS institutions during the 2000-2019 time period. The first sort was to identify first-time degree-seeking students enrolled at VCCS institutions. Students who were enrolled exclusively through dual enrollment programs while they were in high school were eliminated. Students who had earned a post-secondary degree prior to enrolling at VCCS were eliminated as well. These sorts eliminated 15 percent of all students.
The researchers further reduced the pool of students by focusing on students who were enrolled at VCCS institutions between the 2009-2010 and 2013-2014 academic years. This period was selected in order to observe academic and employment data about the students for a period before and after their enrollment. The dataset was further reduced by excluding students who had not earned a degree from a non-VCCS institution and by also including only students between the ages of 18 and 50. These restrictions reduced the dataset to a sample of 376.366 students who earned some credits but not a degree during this period and who dropped out of the VCCS school without returning for at least three years. All these students are referred to as SCND (some college, no degree) at various times in the paper and in this writeup.
The researchers further reduced the dataset of 376,366 students to 95,380 students by focusing on students who earned at least 30 college credits and whose cumulative GPA was at least a 2.0 as of the term immediately before leaving or graduating. Their logic is that these are the students who have shown that they can progress in college and made significant progress toward completing a degree. If you subtract the 95,380 students from the beginning group of 376,366 students, you’ll note that 280,986 students did not complete 30 credits during a five- year period which they may have been enrolled. That reflects a 75 percent failure to complete rate that is not analyzed in this report.
The population of 95,380 students is further parsed into three groups based on their status when they left VCCS: Group A – students who left VCCS without earning a degree or transferring to a non-VCCS school (30.8%), Group B – students who transferred directly to a non-VCCS school without taking a term off (18.9%), and Group C – students who earned a VCCS degree without taking a term off (50.3%).
Group A is further reviewed for those students who may have reenrolled at some point in time in five one-year intervals (years one through five). Approximately 34 percent reenrolled within a year of their break, most at VCCS institutions. Among the group that reenrolled within a year, 50.5 percent attending VCCS institutions completed and 53.5 percent attending non-VCCS institutions completed. A much smaller percentage reenrolled in the second year after a break with 12.3 percent enrolling at a VCCS school and 3.5 percent enrolling at a non-VCCS school. For students reenrolling at a VCCS school the number decreases to 4.0 percent in the third year and 1.3 percent in the fifth year following a break. More than a third (38.9%) never reenrolled anywhere within five years after their break from VCCS.
The researchers determined that the group of students that they included in their final analysis were students who left VCCS between the summer of 2009 and the spring of 2014 with at least 30 credits and a minimum GPA of 2.0 and who did not reenroll anywhere for a period of three years after leaving. This reduced the sample size to n = 26,031. The students who did not meet the 30 credits and the GPA were n = 194,313. Less than 12 percent of the community college population in Virginia made substantial progress toward their degree or fewer than one out of eight students. It’s interesting that the authors chose to word this as less than one in seven which is true but not an indicator of how low it really is.
For some of the analyses, the researchers compared the group of 26,031 to a group of students who earned their last VCCS degree between the 2009-2010 and 2013-2014 academic years and had no subsequent enrollment at a VCCS institution or a non-VCCS institution within three years of graduation (n = 28,795).
All key analyses for these two groups were repeated after separating each sample into younger (24 years old or younger) and older (25 and older) students based on their age at the time they left VCCS since academic and employment outcomes might be different based on the age and work experience of the student.
Earnings measurements were grouped into three categories. The first category was whether the student was employed at all. The second was average quarterly wages, conditional on employment. If the student was not employed during that year, the measure was set to missing. The third measure was whether the student’s earnings were above 200 percent of the federal poverty level threshold (which was set as a two-person household with no children). If a student was not employed during the year, the measure was set to missing. The researchers were primarily interested in the non-zero quarterly wages in the fifth year following the student’s break from VCCS since this is the furthest period analyzed for these cohorts. Of note is that the researchers could not observe whether a student with no earnings was unemployed or was in one of the groups that the VEC does not track (self-employed, federal workers, and out-of-state employers).
Observations for differences in earnings of the SCND students versus graduates were estimated for programs of study with at least 100 combined SCND and Graduates with at least 10 observations in each separate category. Approximately 40 percent of the SCND group received a Pell Grant and 15.6 percent received a Stafford loan in the year prior to their break compared to 36.8 percent and 13.1 percent for graduates. Debt accumulation at the time of students’ break was approximately $3,500 for both samples. Graduates accumulated 62.5 credits vs. 47.7 credits for the SCND group (note – this average for the SCND group is higher than I estimated with a minimum threshold of 30 credits).
The researchers went to great lengths to parse and sort the data looking for trends. From my perspective, the following thoughts surfaced:
- Most community college students who earn some college credit never make it to the criteria of substantially completed (30 credit hours or more plus a 2.0 GPA or greater). Studying that very large group would have taken the researchers down a rabbit hole with a myriad of explanations or attempted explanations of why these students dropped out.
- Of the 95,380 students enrolled during the five-year period studied, 50.3 percent earned a VCCS degree without taking a break (including the summers). A solid piece of advice to all open enrollment institutions, keep your serious students engaged and remind them to enroll for the next term.
- Approximately 40 percent of the study group (Group A) (38.9%) never returned to college. Of those that returned, the group with the highest completion rate were the students who returned within a year. Once again, keep your students engaged.
- With three of the four researchers based out of the University of Virginia, they were able to gain access to the VEC records of a large percentage of the students. The researchers note that this data did not include earnings for the self-employed, federal workers, or those employed by out-of-state employers. In certain areas of the VCCS, notably Northern Virginia and the Tidewater area of Norfolk, there are higher than average numbers of federal workers including active-duty military as well as major employers who are considered out-of-state. Knowing this percentage would be important for accountability purposes of individual institutions, especially if it appears that a large number of former students or graduates are “unemployed” because their earnings register as zero in the VEC records. At least it’s better than the College Scorecard which does not include earnings at the program level for students who do not borrow.
- The researchers observed that a high percentage of dropouts experienced a significant drop in their GPA and credits earned in the last semester that they attended. Specifically, the SCND students had a mean GPA of 2.93 in the years preceding their break whereas in the term immediately preceding their break, their mean GPA declined substantially to 2.34. Focusing on completed credits, the SCND group completed an average of 7 credits per term preceding their last term. In the term immediately preceding their break, they completed a mean of 4.9 completed credits (note: the researchers parsed the SCND group in three segments for further GPA analysis – those students with no disruption in grades [14,845], those with a GPA drop of greater than one percent [4,615], and those who received grades of W/F for all credits attempted [6,571]. The researchers observed that this was a sizable percentage of students that received a Pell Grant or borrowed federal loans in their break term (36% of Pell Grant recipients received no grant during their break term and 34% of federal loan recipients received no loan during their break term). They note that a student must be considered at least attending half time to be eligible for Federal Student Aid. When a student goes below half time status, they are obligated to return the federal funds (grants and loans) that they received. In almost all cases, the colleges must return the federal funds and attempt to collect the payment overage from the students who likely don’t have the money anymore. Students with balances due are generally not allowed to reenroll until the balances are repaid and their transcripts are not forwarded to other institutions. This area is worthy of additional research.
- As online classes and programs have become more available because of the pandemic, it would be useful to see if there are any colleges in the VCCS group that could parse their students between those studying online and those studying on campus. If the campus online offerings were more frequent than the traditional three term year, it would be reasonable to see if more frequent course offerings enabled more adults to stay enrolled.
The authors’ research enables them to conclude that the share of students who are academically prepared, earned credits at community colleges, did not graduate, and left with a GPA of 2.0 or greater is much smaller than what policy makers estimate. As reported in the Brookings piece, the authors also note that most of the degrees that they analyzed did not provide a significant earnings premium for graduates versus the SCND population. They estimated that only three percent of the SCND population could easily reenroll in fields of study from which they could expect an earnings premium.
I believe that transparency in earnings outcomes for college graduates is extremely important. Initiatives like this paper go a long way toward attempting to develop granularity in the earnings outcomes of students while also exposing where there are weaknesses. In Virginia, the VEC doesn’t capture data from all workers. This is almost as flawed as the College Scorecard’s refusal to include data from all students (they only include data from students who participate in the Federal Student Aid program). In the study, the researchers acknowledge that there are large numbers of “unemployed” students or graduates based on the $0 earnings received from VEC that may be in the excluded classes. They determined to focus on the earnings numbers received because that data did not exclude students who did not borrow money from their analysis.
America’s community colleges have experienced enrollment declines during the pandemic. I’m curious if this study (and hopefully others to come) can surface some solutions to the decline like engaging students more before they experience an academic or financial disruption and providing them with program pathways that de-risk the chances that they experience a difficult course after completing more than half their program.
