In a recently issued paper, co-authors Karen Singer-Freeman and Christine Robinson report the results of their project to identify the grand challenges for assessment in higher education.
In order to provide a perspective oriented toward the future, the researchers reviewed assessment websites, blogs, discussion boards, and publications from 2015-2019. The materials reviewed included 83 unique writings; 34 were peer-reviewed, 46 were non-peer-reviewed, and 3 were blog or discussion board posts.
From their review, ten potential challenges were identified that filled the four defining characters of grand challenges (if this sounds familiar, over the summer, I reviewed the Grand Challenges in higher education that could be resolved through a digital transformation, as published by Educause. The ten challenges identified were included in a national survey of higher education assessment professionals. Four of the challenges with broad support were selected as a starting point for strategic planning and collective problem solving from nine national organizations.
The Ten Grand Challenges in Assessment
Drive Innovation – The process of assessment should produce visible and actionable assessment findings that drive innovation. Ms. Singer-Freeman and Ms. Robinson write that there has been a notable increase in the use of data-driven decision making, but there is no evidence that data-driven decisions have improved students’ experiences or outcomes. Per the authors, to effectively drive innovation, it is necessary to improve assessment methodology so that the evidence gathered informs an understanding of the outcomes associated with innovative practices. This challenge is one of the four included in the Grand Challenges in Assessment Project.
Inform Budget – Assessment findings should be used to inform budgetary decisions. Performance-based funding for state colleges and universities has been identified as a top issue facing higher education. Tools used to analyze return on investment can be utilized to determine which student success initiatives provide value beyond their costs. The authors encourage institutions to take steps to integrate planning, assessment, and budgeting.
Immediate Improvements – Assessment findings should be used to direct immediate pedagogical improvements. Too often findings are not utilized in time to benefit the students being assessed. We must find ways to make changes in response to assessment findings within the space of a single class through formative assessments. Per the authors, the rapid increase in online teaching and adaptive learning provides opportunities for assessments to take place in real-time. Information about student cognitive skills, social-emotional development, and current academic accomplishments can be used by faculty to determine how their pedagogy is impacting current students. This challenge is one of the four challenges with broad support.
Increase equity – Assessment findings should be used to increase educational equity. Assessments must be designed and analyzed to determine which institutions of higher education are providing access to higher education for all students. To effectively increase equity in higher education, the authors write that we must increase our use of data disaggregation. This challenge is the one with the broadest support and is included in the group of Four Grand Challenges.
Disaggregate Data – Data on learning should be disaggregated to consider important student characteristics. Most assessment data are reported and reviewed in ways that mask inequities because they are reported across all sections of courses and instructors. As the disaggregation of data increases, the authors recommend careful consideration of which groupings of students are appropriate.
Change Over Time – To identify progress, it is essential to examine changes in institutional effectiveness (including student learning) over time. The authors note that strategic planning in business effectively supports continuous improvement because of rigorous follow-up. They note that there is limited follow-up to higher education strategic planning. To maximize the use of strategic planning including institutional effectiveness, there must be tracking of the achievement of those outcomes over time. In order to track outcomes over time, technology providing longitudinal student data must be utilized, i.e. ePortfolios. This challenge was one of the four with the broadest support and is included in the Grand Challenges in Assessment Project.
Student Self-Evaluation – Involving students in authentic self-evaluation of their own learning enhances the learning process. Competency-based education or e-Portfolios are ways to encourage students to evaluate their learning and increase their awareness of their accomplishments. The authors write that to fully integrate self-evaluation of learning into pedagogy, it will be important to provide faculty with training on best practices in evaluative student reflections. Increasing student self-evaluations can improve both student learning and the assessment of student learning.
ePortfolios – Increasing the use of ePortfolios can capture student learning over the entire span of their educational career. With an ePortfolio, students can capture examples of their work, reflect on their progress toward personal and educational goals, and share their work with faculty, family, and future employers. The authors write that the heart of the ePortfolio practice is the documentation of and reflection on learning. A high-quality ePortfolio curates curricular and co-curricular experiences and has the potential to document the full range of learning that students experience over time and at different institutions.
Massive Data – It is time to leverage technology to analyze massive datasets within and across institutions. Evolving technologies (such as MOOCS, Learning Management Systems (LMS), artificial intelligence platforms, and adaptive learning platforms) now collect large datasets with a wide range of variables including performance on individual assignments, time spent on tasks, and utilization of linked resources. These datasets provide institutions with opportunities to look at learning in more ways than previously available. Critically, those institutions that would like to collaborate and share data, data systems, and assessment personnel, would need to reach a consensus on the common metrics utilized and how the findings will be used.
Communicate – Effectively communicating relevant, timely, and contextualized information about the full range of experiences to stakeholders will contribute to student learning and success. An important point that the authors make is “with attention to the needs of different audiences, we must begin to offer clear information about why we provide students with different experiences in higher education and how those specific experiences benefit students.” A quick review of earnings data by degree now included on the College Scorecard website could shift students’ choices of study. Transparent communications about the outcomes of individual programs for an institution’s graduates could go a long way to prevent misunderstandings by families, external regulatory bodies, and politicians. Progress toward the other nine challenges will need to be effectively communicated as well in order to identify solutions, generate funding, and coordinate progress toward goals.
Ms. Singer-Freeman and Ms. Robinson conclude their paper by writing that solutions to these grand challenges will require shifting the role of assessment from a compliance-oriented role towards improvement-oriented. Based on the results of their survey distributed to the assessment professionals, they believe they are ready to use assessment to increase equity, innovation, pedagogy, and document progress over time. I believe that they also could have included that evolving assessments from a compliance-oriented role toward an improvement-oriented role will enable institutions to be much more market savvy. Market savvy institutions could partner with employers to design assessments that measure the progress of students in specific areas important to that employer. An improvement-focused institution would embrace such a role. The authors believe that the assessment community can’t solve these challenges alone which is why using the grand challenge concept is a way to reach out to the broader community utilizing assessments.
Kudos to all of those who participated in Ms. Singer-Freeman’s and Ms. Robinson’s research! With many institutions having maintained their classes online throughout the pandemic, there is more data than ever before that could be utilized for learning assessment. Institutions that have a large number of online students have been able to track all of the metrics called for by the authors. I don’t see where there are competitive issues that would prevent collaboration across institutions, particularly large online institutions that have collected the data for years. It’s time we make a change and these grand challenges are capable of being solved if there is greater cooperation.
