Home Accountability Grand Challenges in Assessment: Collective Issues in Need of Solutions
Grand Challenges in Assessment: Collective Issues in Need of Solutions

Grand Challenges in Assessment: Collective Issues in Need of Solutions

0

In a recently issued paper, co-authors Karen Singer-Freeman and Christine Robinson report the results of their project to identify the grand challenges for assessment in higher education.

In order to provide a perspective oriented toward the future, the researchers reviewed assessment websites, blogs, discussion boards, and publications from 2015-2019. The materials reviewed included 83 unique writings; 34 were peer-reviewed, 46 were non-peer-reviewed, and 3 were blog or discussion board posts.

From their review, ten potential challenges were identified that filled the four defining characters of grand challenges (if this sounds familiar, over the summer, I reviewed the Grand Challenges in higher education that could be resolved through a digital transformation, as published by Educause. The ten challenges identified were included in a national survey of higher education assessment professionals. Four of the challenges with broad support were selected as a starting point for strategic planning and collective problem solving from nine national organizations.

The Ten Grand Challenges in Assessment

Drive Innovation – The process of assessment should produce visible and actionable assessment findings that drive innovation. Ms. Singer-Freeman and Ms. Robinson write that there has been a notable increase in the use of data-driven decision making, but there is no evidence that data-driven decisions have improved students’ experiences or outcomes. Per the authors, to effectively drive innovation, it is necessary to improve assessment methodology so that the evidence gathered informs an understanding of the outcomes associated with innovative practices. This challenge is one of the four included in the Grand Challenges in Assessment Project.

Inform Budget – Assessment findings should be used to inform budgetary decisions. Performance-based funding for state colleges and universities has been identified as a top issue facing higher education. Tools used to analyze return on investment can be utilized to determine which student success initiatives provide value beyond their costs. The authors encourage institutions to take steps to integrate planning, assessment, and budgeting.

Immediate Improvements – Assessment findings should be used to direct immediate pedagogical improvements. Too often findings are not utilized in time to benefit the students being assessed. We must find ways to make changes in response to assessment findings within the space of a single class through formative assessments. Per the authors, the rapid increase in online teaching and adaptive learning provides opportunities for assessments to take place in real-time. Information about student cognitive skills, social-emotional development, and current academic accomplishments can be used by faculty to determine how their pedagogy is impacting current students. This challenge is one of the four challenges with broad support.

Increase equity – Assessment findings should be used to increase educational equity. Assessments must be designed and analyzed to determine which institutions of higher education are providing access to higher education for all students. To effectively increase equity in higher education, the authors write that we must increase our use of data disaggregation. This challenge is the one with the broadest support and is included in the group of Four Grand Challenges.

Disaggregate Data – Data on learning should be disaggregated to consider important student characteristics. Most assessment data are reported and reviewed in ways that mask inequities because they are reported across all sections of courses and instructors. As the disaggregation of data increases, the authors recommend careful consideration of which groupings of students are appropriate.

Change Over Time – To identify progress, it is essential to examine changes in institutional effectiveness (including student learning) over time. The authors note that strategic planning in business effectively supports continuous improvement because of rigorous follow-up. They note that there is limited follow-up to higher education strategic planning. To maximize the use of strategic planning including institutional effectiveness, there must be tracking of the achievement of those outcomes over time. In order to track outcomes over time, technology providing longitudinal student data must be utilized, i.e. ePortfolios. This challenge was one of the four with the broadest support and is included in the Grand Challenges in Assessment Project.

Student Self-Evaluation – Involving students in authentic self-evaluation of their own learning enhances the learning process. Competency-based education or e-Portfolios are ways to encourage students to evaluate their learning and increase their awareness of their accomplishments. The authors write that to fully integrate self-evaluation of learning into pedagogy, it will be important to provide faculty with training on best practices in evaluative student reflections. Increasing student self-evaluations can improve both student learning and the assessment of student learning.

ePortfolios – Increasing the use of ePortfolios can capture student learning over the entire span of their educational career. With an ePortfolio, students can capture examples of their work, reflect on their progress toward personal and educational goals, and share their work with faculty, family, and future employers. The authors write that the heart of the ePortfolio practice is the documentation of and reflection on learning. A high-quality ePortfolio curates curricular and co-curricular experiences and has the potential to document the full range of learning that students experience over time and at different institutions.

Massive Data – It is time to leverage technology to analyze massive datasets within and across institutions. Evolving technologies (such as MOOCS, Learning Management Systems (LMS), artificial intelligence platforms, and adaptive learning platforms) now collect large datasets with a wide range of variables including performance on individual assignments, time spent on tasks, and utilization of linked resources. These datasets provide institutions with opportunities to look at learning in more ways than previously available. Critically, those institutions that would like to collaborate and share data, data systems, and assessment personnel, would need to reach a consensus on the common metrics utilized and how the findings will be used.

Communicate – Effectively communicating relevant, timely, and contextualized information about the full range of experiences to stakeholders will contribute to student learning and success. An important point that the authors make is “with attention to the needs of different audiences, we must begin to offer clear information about why we provide students with different experiences in higher education and how those specific experiences benefit students.” A quick review of earnings data by degree now included on the College Scorecard website could shift students’ choices of study. Transparent communications about the outcomes of individual programs for an institution’s graduates could go a long way to prevent misunderstandings by families, external regulatory bodies, and politicians. Progress toward the other nine challenges will need to be effectively communicated as well in order to identify solutions, generate funding, and coordinate progress toward goals.

Ms. Singer-Freeman and Ms. Robinson conclude their paper by writing that solutions to these grand challenges will require shifting the role of assessment from a compliance-oriented role towards improvement-oriented. Based on the results of their survey distributed to the assessment professionals, they believe they are ready to use assessment to increase equity, innovation, pedagogy, and document progress over time. I believe that they also could have included that evolving assessments from a compliance-oriented role toward an improvement-oriented role will enable institutions to be much more market savvy. Market savvy institutions could partner with employers to design assessments that measure the progress of students in specific areas important to that employer. An improvement-focused institution would embrace such a role. The authors believe that the assessment community can’t solve these challenges alone which is why using the grand challenge concept is a way to reach out to the broader community utilizing assessments.

Kudos to all of those who participated in Ms. Singer-Freeman’s and Ms. Robinson’s research! With many institutions having maintained their classes online throughout the pandemic, there is more data than ever before that could be utilized for learning assessment. Institutions that have a large number of online students have been able to track all of the metrics called for by the authors. I don’t see where there are competitive issues that would prevent collaboration across institutions, particularly large online institutions that have collected the data for years. It’s time we make a change and these grand challenges are capable of being solved if there is greater cooperation.

Comments

comments

Wally Boston Dr. Wallace E. Boston was appointed President and Chief Executive Officer of American Public University System (APUS) and its parent company, American Public Education, Inc. (APEI) in July 2004. He joined APUS as its Executive Vice President and Chief Financial Officer in 2002. In July 2016, he retired as APUS president and continued as CEO of APEI. In September 2017, he was reappointed APUS president after the resignation of Dr. Karan Powell. In September 2019, Angela Selden was named CEO of APEI, succeeding Dr. Boston who will remain APUS president until his planned retirement in June 2020. Dr. Boston guided APUS through its successful initial accreditation with the Higher Learning Commission of the North Central Association in 2006 and ten-year reaccreditation in 2011. In November 2007, he led APEI to an initial public offering on the NASDAQ Exchange. During his tenure, APUS grew to over 100,000 students, 200 degree and certificate programs, and approximately 90,000 alumni. In addition to his service as a board member of APUS and APEI, Dr. Boston is a member of the Board of Advisors of the National Institute for Learning Outcomes Assessment (NILOA), a member of the Board of Overseers of the University of Pennsylvania’s Graduate School of Education, a board member of the Presidents’ Forum, and a board member of Hondros College of Nursing and Fidelis, Inc. He has authored and co-authored papers on the topic of online post-secondary student retention, and is a frequent speaker on the impact of technology on higher education. Dr. Boston is a past Treasurer of the Board of Trustees of the McDonogh School, a private K-12 school in Baltimore. In his career prior to APEI and APUS, Dr. Boston served as either CFO, COO, or CEO of Meridian Healthcare, Manor Healthcare, Neighborcare Pharmacies, and Sun Healthcare Group. Dr. Boston is a Certified Public Accountant, Certified Management Accountant, and Chartered Global Management Accountant. He earned an A.B. degree in History from Duke University, an MBA in Marketing and Accounting from Tulane University’s Freeman School of Business Administration, and a Doctorate in Higher Education Management from the University of Pennsylvania’s Graduate School of Education. In 2008, the Board of Trustees of APUS awarded him a Doctorate in Business Administration, honoris causa, and, in April 2017, also bestowed him with the title President Emeritus. Dr. Boston lives in Owings Mills, MD with his wife Sharon and their two daughters.

LEAVE YOUR COMMENT

Your email address will not be published. Required fields are marked *