Another New Undergraduate Ranking System – Substance over Form?

Last week, University Business published an article titled The 50 institutions that offer the best starting salaries for 4-year degree holders. Written by associate editor Chris Burt, the article outlines the 2022 results for a ranking system developed by Optimal, a data-driven company dedicated to enabling people to make better decisions about careers and education.

Optimal developed a product called GradReports that sources data from the College Scorecard. From this tool, Optimal created the 2021 and 2022 Best Colleges by Salary Score. Each college’s Salary Score is based on a comparison of median first year salaries post-graduation (this will not reflect the salaries for all alumni since the College Scorecard only tracks students who borrowed money through the Federal Student Aid program) across the same programs at all schools weighted by student enrollment in each program (also data reported from the College Scorecard and also data that is only reported for students who borrow – to be fair to Optimal, they report this shortcoming in their methodology under limitations of the data).

Optimal matches the salary data by degree across institutions using the CIP codes tracked by the College Scorecard. To be eligible for the bachelor’s degree rankings, schools need to have at least 10 CIP codes with reportable earnings (eight programs are required for the master’s rankings and five programs are required for the associate degree rankings). To calculate a school’s overall Salary Score by level, z-scores for each program with a reported CIP program were calculated. An average of the z-scores for all the programs at each school by level was then calculated. This average z-score was converted to a Salary Score for each school.

I visited the Optimal website and while the company does provide a link to its methodology (as cited above), the 2022 results reported by University Business were located on a sister website at OnlineU.com. I compiled a list of the 2022 Top 30 colleges and included it below.

ONLINEU.COM 2022 Best Undergraduate Colleges in the U.S. as Ranked by Alumni Salaries
College/University 2022 Ranking 2021 Ranking Change Annual Tuition Salary Score
Harvard University 1 1 0 51,925 99.51
MIT 2 2 0 53,790 99.48
Dartmouth College 3 5 2 57,638 98.89
Yale University 4 3 -1 55,500 98.71
Georgetown University 5 6 1 56,058 98.35
Carnegie Mellon University 6 8 2 57,119 98.3
University of Pennsylvania 7 4 -3 57,770 98.15
Santa Clara University 8 7 -1 53,634 97.66
Boston College 9 11 2 57,910 97.58
Stanford University 10 9 -1 53,529 96.99
Columbia University 11 19 8 61,788 96.72
Bentley University 12 15 3 51,830 96.67
Colgate University 13 18 5 58,045 96.45
Thomas Edison State University 14 20 6 9,856 95.83
Williams College 15 34 19 57,280 95.52
Princeton University 16 13 -3 52,800 95.28
University of California – Berkeley 17 22 5 44,007 94.25
Barnard College 18 12 -6 57,668 94.23
Bucknell University 19 25 6 58,196 94.11
University of Chicago 20 32 12 60,552 93.65
University of San Francisco 21 29 8 50,282 93.51
University of Notre Dame 22 21 -1 55,553 93.14
Cornell University 23 31 8 57,222 92.81
Northeastern University 24 23 -1 53,506 92.73
Lehigh University 25 33 8 55,240 92.52
American Public University System 26 42 16 7,324 92.15
Brown University 27 50 23 58,404 91.89
Northwestern University 28 24 -4 56,691 91.88
National University 29 50 21 13,320 91.77
Capella University 30 35 5 14,540 91.76

Since the rankings are based on Optimal’s calculated Salary Score, any change in data reported by the College Scorecard will trigger a change in the ranking. Brown University and National University were not included in the Top 50 rankings last year. Because I was unable to determine their ranking for last year, I gave them each a ranking of 50.

The Top 10 colleges did not move around much between last year and this year. The next 10 (11 through 20) showed a little more movement with Williams College improving by 19 slots. Included in the next 10 were the first two public universities, Thomas Edison State University at 14 and the University of California – Berkeley at 17. I noted that there is a substantial tuition price difference between the two.

The final 10 colleges (21 through 30) include two online, for-profit universities, American Public University System (APUS) at 26 and Capella University at 30. Both of these universities were in the top two percent of all colleges and universities for Return on Investment (ROI) as measured by Georgetown’s Center on Education and the Workforce (my critique published here) so seeing them in the top 30 based on alumni earnings is not a surprise.

While I like the fact that the university I led for 18 years (APUS) is included in Optimal’s Top 30, I’m not ready to declare the Optimal rating system ready for prime time. Optimal’s methodology clearly outlines the risk of using College Scorecard data to drive their rankings. I think their normalization of comparisons through z-scores and CIP codes is sound, although it assumes that the degrees included in CIP codes are apples to apples which may not be the case. Optimal also uses the first-year earnings for alumni and that could be distorted by graduate school attendance, lower paid pre-grad school internships or something else. The Department has recognized this issue and continues to add an additional year’s salary data to the College Scorecard each year it is updated.

Even though a minimum of 10 programs with earnings data in the College Scorecard are required for participation in the bachelor’s degree evaluation by Optimal, the fact that the Department masks data for degrees with less than 25 annual graduates (exact number is not disclosed) means that many degrees and many schools are not included in the evaluation (Optimal discloses that approximately 900 colleges are included in the bachelor’s rankings, far less than participate in Federal Student Aid programs).

Even though programs are weighted by enrollments, it’s somewhat random which programs will end up with the higher salary ranking across all colleges and universities. Taking an average of the z-scores for each institution may be reasonable since they’re weighted for enrollment, but enrollments of only students who borrow could be misleading for certain degrees, particularly those with large groups of employer-reimbursed students.

It’s clear to me that Optimal uses these rankings (and others) to provide marketing and other services to colleges and universities. At the same time, so does U.S. News and World Report. There is far less subjectivity in Optimal’s rankings than U.S. News, but if I led a school that wanted to issue a press release about the Optimal ranking, I would avoid using specific earnings numbers published by Optimal and/or the College Scorecard.

Will we ever see a perfect college rating/ranking system? I doubt it. Can they get better? Perhaps, but until Congress authorizes a unified student data system, data from colleges and universities published in the College Scorecard will not be accurate.

I haven’t tried to count the various college ratings that exist, but there are many. Are there too many? Probably. At the same time, every new ratings schema that surfaces likely does because someone is trying to find a better way to compare and rank colleges and universities.

Lastly, I note that even if the perfect system for ranking colleges and universities existed, there would be some colleges whose student enrollments would exceed expectations based on their ranking. Spending more money on marketing than your competition works in many industries. Higher education is no exception.

Subjects of Interest

EdTech

Higher Education

Independent Schools

K-12

Student Persistence

Workforce