Home Business of Education The Ongoing Debate About EdTech Evidence
The Ongoing Debate About EdTech Evidence

The Ongoing Debate About EdTech Evidence

0

A little more than a year ago, I wrote about an EdSurge article asking if the expenditures for EdTech in K-12 were wasteful. Senior Reporter Emily Tate authored that article and has written another article about the edtech industry asking the question “are companies ready to provide [evidence] that edtech works as promised.”

Ms. Tate Sullivan opens her recent article with a statement that schools are awash in technology thanks to the scramble to go online during the pandemic. Whether the technology works to improve outcomes for kids is unknown since research to prove it has not been conducted. With approximately 9,000 edtech products on the market, Ms. Tate Sullivan writes that the landscape may be changing with school district leaders asking for the evidence indicating that the products work as promised.

An example of a company that believes in research is Bamboo Learning. Bamboo Learning offers a voice-enabled literacy application for children in kindergarten through 5th grade. As soon as they shifted their product focus from parents to schools, Ms. Tate Sullivan writes that Bamboo’s co-founder, a former educator, wanted to have a product informed by research and focus groups.

Bamboo worked with LearnPlatform, an edtech effectiveness system that helps school districts manage their edtech products. The goal for Bamboo was to demonstrate that they complied with the first level of evidence (Tier 4: Demonstrates a rationale – see illustration below) under the Every Student Succeeds Act passed by Congress in 2015.

Ms. Tate Sullivan notes that Tier 4 “is not a high bar.” It requires companies to show a logic model and have plans to study the effects of the product. Bamboo received their Tier 4 certification in February. After receiving Tier 4 certification, Bamboo pursued the Tier 3 certification which required a minimum of one “well-designed and well-implemented correlational study with statistical controls.” The study showed that the students who used the Bamboo application demonstrated improved reading and listening comprehension skills as well as high levels of engagement. Learn Platform published the outcomes of that study and Bamboo earned its Tier 3 certification. Bamboo intends to pursue a Tier 2 certification which requires a study with a 300-student sample size.

There has already been a payoff for Bamboo. This spring, the company was one of 200 vendors that responded to an RFP from a school district with 12,000 K-5 students. The logic model from Tier 4 and the expected learning outcomes from the Tier 3 study was shared with the decision-making committee. Bamboo was awarded the contract.

Despite this example where an edtech company provided evidence of successful student outcomes and won an RFP, Ms. Tate Sullivan writes that most companies do not pursue independent, rigorous research using their products because they don’t have to. She interviewed Bart Epstein, CEO of the EdTech Evidence Exchange, who stated that one of the great flaws in the edtech industry is that there are few barriers to entry and no governing body holds companies accountable for their claims. “It’s so easy to game the system,” notes Mr. Epstein. “Decisions about what to purchase are far more often made on usability, personal relationships, features, and not on evidence.”

LearnPlatform thinks that they might be part of the solution. Their evidence-as-a-service model makes third-party evaluation available to edtech providers at a fraction of the cost and time. It is also delivered to districts in a more accessible format. Their CEO stated that when sales loses an RFP to a company that demonstrated it had evidence through a bonafide study, more companies will take the time to assemble their own research studies for efficacy.

Ms. Tate Sullivan’s article is timely. As a decision-maker on many edtech products over the past two decades, I cannot remember a single vendor who provided us with an independently researched and correlated study. At the same time, whenever we requested references from satisfied users, we were given contact information for people who would be glad to answer our questions over the phone or even meet with us to discuss the effectiveness of the product.

The IT industry in general has often marketed its products based more on appearance and who else is using it. For many of those products, the appeal was related to back-office efficiency or front desk/sales counter efficiency. As more and more products surface that are designed to improve students’ learning outcomes, I would advise decision-makers to insist on bonafide efficacy studies. I would also recommend company leaders to pursue the certification route. Options like LearnPlatform remove the need to locate statisticians and academics who can design an appropriate study that survives the tests of reliability. I look forward to hearing about more successes such as Age of Learning, developer of My Math Academy, that has earned ESSA Level I certification for its product.

Wally Boston Dr. Wallace E. Boston was appointed President and Chief Executive Officer of American Public University System (APUS) and its parent company, American Public Education, Inc. (APEI) in July 2004. He joined APUS as its Executive Vice President and Chief Financial Officer in 2002. In September 2019, Dr. Boston retired as CEO of APEI and retired as APUS President in August 2020. Dr. Boston guided APUS through its successful initial accreditation with the Higher Learning Commission of the North Central Association in 2006 and ten-year reaccreditation in 2011. In November 2007, he led APEI to an initial public offering on the NASDAQ Exchange. For four years from 2009 through 2012, APEI was ranked in Forbes' Top 10 list of America's Best Small Public Companies. During his tenure as president, APUS grew to over 85,000 students, 200 degree and certificate programs, and approximately 100,000 alumni. While serving as APEI CEO and APUS President, Dr. Boston was a board member of APEI, APUS, Hondros College of Nursing, and Fidelis, Inc. Dr. Boston continues to serve as a member of the Board of Advisors of the National Institute for Learning Outcomes Assessment (NILOA) and as a member and chair of the board of New Horizons Worldwide. He has authored and co-authored papers on the topic of online post-secondary student retention, and is a frequent speaker on the impact of technology on higher education. Dr. Boston is a past Treasurer of the Board of Trustees of the McDonogh School, a private K-12 school in Baltimore. In his career prior to APEI and APUS, Dr. Boston served as either CFO, COO, or CEO of Meridian Healthcare, Manor Healthcare, Neighborcare Pharmacies, and Sun Healthcare Group. Dr. Boston is a Certified Public Accountant, Certified Management Accountant, and Chartered Global Management Accountant. He earned an A.B. degree in History from Duke University, an MBA in Marketing and Accounting from Tulane University’s Freeman School of Business Administration, and a Doctorate in Higher Education Management from the University of Pennsylvania’s Graduate School of Education. In 2008, the Board of Trustees of APUS awarded him a Doctorate in Business Administration, honoris causa, and, in April 2017, also bestowed him with the title President Emeritus. In August 2020, the Board of Trustees of APUS appointed him Trustee Emeritus. In November 2020, the Board of Trustees announced that the APUS School of Business would be renamed the Dr. Wallace E Boston School of Business in recognition of Dr. Boston's service to the university. Dr. Boston lives with his family in Austin, Texas.

LEAVE YOUR COMMENT

Your email address will not be published.