A little more than a year ago, I wrote about an EdSurge article asking if the expenditures for EdTech in K-12 were wasteful. Senior Reporter Emily Tate authored that article and has written another article about the edtech industry asking the question “are companies ready to provide [evidence] that edtech works as promised.”
Ms. Tate Sullivan opens her recent article with a statement that schools are awash in technology thanks to the scramble to go online during the pandemic. Whether the technology works to improve outcomes for kids is unknown since research to prove it has not been conducted. With approximately 9,000 edtech products on the market, Ms. Tate Sullivan writes that the landscape may be changing with school district leaders asking for the evidence indicating that the products work as promised.
An example of a company that believes in research is Bamboo Learning. Bamboo Learning offers a voice-enabled literacy application for children in kindergarten through 5th grade. As soon as they shifted their product focus from parents to schools, Ms. Tate Sullivan writes that Bamboo’s co-founder, a former educator, wanted to have a product informed by research and focus groups.
Bamboo worked with LearnPlatform, an edtech effectiveness system that helps school districts manage their edtech products. The goal for Bamboo was to demonstrate that they complied with the first level of evidence (Tier 4: Demonstrates a rationale – see illustration below) under the Every Student Succeeds Act passed by Congress in 2015.
Ms. Tate Sullivan notes that Tier 4 “is not a high bar.” It requires companies to show a logic model and have plans to study the effects of the product. Bamboo received their Tier 4 certification in February. After receiving Tier 4 certification, Bamboo pursued the Tier 3 certification which required a minimum of one “well-designed and well-implemented correlational study with statistical controls.” The study showed that the students who used the Bamboo application demonstrated improved reading and listening comprehension skills as well as high levels of engagement. Learn Platform published the outcomes of that study and Bamboo earned its Tier 3 certification. Bamboo intends to pursue a Tier 2 certification which requires a study with a 300-student sample size.
There has already been a payoff for Bamboo. This spring, the company was one of 200 vendors that responded to an RFP from a school district with 12,000 K-5 students. The logic model from Tier 4 and the expected learning outcomes from the Tier 3 study was shared with the decision-making committee. Bamboo was awarded the contract.
Despite this example where an edtech company provided evidence of successful student outcomes and won an RFP, Ms. Tate Sullivan writes that most companies do not pursue independent, rigorous research using their products because they don’t have to. She interviewed Bart Epstein, CEO of the EdTech Evidence Exchange, who stated that one of the great flaws in the edtech industry is that there are few barriers to entry and no governing body holds companies accountable for their claims. “It’s so easy to game the system,” notes Mr. Epstein. “Decisions about what to purchase are far more often made on usability, personal relationships, features, and not on evidence.”
LearnPlatform thinks that they might be part of the solution. Their evidence-as-a-service model makes third-party evaluation available to edtech providers at a fraction of the cost and time. It is also delivered to districts in a more accessible format. Their CEO stated that when sales loses an RFP to a company that demonstrated it had evidence through a bonafide study, more companies will take the time to assemble their own research studies for efficacy.
Ms. Tate Sullivan’s article is timely. As a decision-maker on many edtech products over the past two decades, I cannot remember a single vendor who provided us with an independently researched and correlated study. At the same time, whenever we requested references from satisfied users, we were given contact information for people who would be glad to answer our questions over the phone or even meet with us to discuss the effectiveness of the product.
The IT industry in general has often marketed its products based more on appearance and who else is using it. For many of those products, the appeal was related to back-office efficiency or front desk/sales counter efficiency. As more and more products surface that are designed to improve students’ learning outcomes, I would advise decision-makers to insist on bonafide efficacy studies. I would also recommend company leaders to pursue the certification route. Options like LearnPlatform remove the need to locate statisticians and academics who can design an appropriate study that survives the tests of reliability. I look forward to hearing about more successes such as Age of Learning, developer of My Math Academy, that has earned ESSA Level I certification for its product.
