Reviewing the Methodology Behind New ROI Rankings for 4,500 Colleges
I am no fan of the Department of Education’s College Scorecard, primarily because it is incomplete and may be misleading for some metrics. Much of the data is derived from students using Federal Student Aid (FSA) only and some of it is from those who are first-time, full-time students using FSA loans. At APUS, most of our students are part-time, working adults not using FSA to fund their education. I first wrote about the Scorecard in 2016 and reported about others like me who criticized its incomplete data.
Despite the flaws of the Scorecard, I understand why Georgetown University’s Center on Education and the Workforce recently attempted to create a return on investment (ROI) for all colleges using this data. First, it’s the only published source that uses IRS data to match earnings with students who have attended those specific institutions and who received FSA. With access to earnings, institutional costs and debt incurred, the researchers can calculate a rudimentary ROI.
For some institutions, using data from FSA students may be representative of all students. In our case, however, this group represents less than half of our active students. Without having access to IRS data from all students, I do not know if the earnings provided are representative or not. The salary for APUS students 10 years after starting is listed at $62,400 in the Scorecard. I assume that one reason why our number is approximately double the median used in the sample of $32,300 is that most of our students are older than 18 when they start and already working full-time. While this number should normally not fluctuate much, I am concerned about the potential impact of students who enrolled in many different online programs from 2010-14 at APUS and other institutions who had no intention of completing their programs or courses, i.e. so-called “stipend chasers”. Unless the Department finds a way to parse this number, it is possible that earnings could decrease in the future if these student earnings are included.
Cost is based on the institution’s net price, which is the average cost of attendance (tuition, fees, books and supplies, and living expenses) minus aid received from all sources. Since we keep tuition affordable and provide textbook grants for undergraduates, our average annual cost in the current Scorecard is $10,284. The debt amount used in the study is the median principal loan amount upon entering repayment for the debt cohort of each institution. While the average loan amount cited for APUS is relatively low at $5,500, it would be closer to $0 if all students were included.
APUS proudly serves U.S. military members. In fact, since 2001, we have capped our undergraduate tuition at the semester hour rate reimbursed by the Department of Defense. In addition, we provide a textbook grant to undergraduates and charge very few, nominal fees. As a result of our founding mission and focus on affordability, we know that currently 72% of all of our graduates do not need to incur loans through APUS to pay for their education.
The Georgetown researchers state that their rankings do not account for a number of factors, including: cohort-, location- and occupation-specific effects, along with industry-specific shocks like a downturn in the home-building industry. I noticed the impact of location-specific effects when comparing APUS’s rankings with other West Virginia institutions. Since most of our students do not live in West Virginia, with one of the lowest per capita incomes, our net present value (NPV) is the highest in the state with only one other institution having a 40-year NPV above $1 million.
Median earnings for students were interpolated using six, eight and 10-year reported earnings, and were not inflated beyond year 10. While this methodology is certainly conservative, it effectively handicaps institutions whose students take longer than four years to complete a bachelor’s degree. The researchers recommend that the Scorecard include data on earnings greater than 10 years, specifically median earnings at 15 and 20 years after enrollment — something I would welcome. For institutions primarily awarding less than four-year degrees, the researchers recommend that the College Scorecard include earnings at shorter intervals such as two, three, and four years after entering. This is also a sound recommendation due to the Department’s methodology of excluding any students still enrolled.
Again, I am glad that the Georgetown researchers published this report on college ROI, and especially to see APUS recognized for fulfilling its mission of providing a quality and affordable education. I hope that the Scorecard expands its data to include all college students even though that may take an act of Congress to facilitate. The most valuable feature of this report is that the researchers have attempted to quantify and evaluate on a relative basis, the ROI generated from many of our U.S. colleges and universities. Those colleges with a high ROI deserve some praise, while those with a low ROI need to examine the factors for that ranking.
There will be major changes in the higher education landscape over the next decade. Institutions that survive and thrive will undoubtedly take into account the demand to provide an ROI to their students.