Guidance for Ed-Tech Companies in Crafting AI-Powered Products

Posted on • Reading Time: 3 min read

This article was written and Published by Alexandria Ng— August 15, 2024

Conceptual user manual illustration of person reading and writing privacy policy and terms and conditions for guide instruction or manual book.

Many districts are struggling to determine what kinds of artificial intelligence products they should allow in their schools.

Ed-tech companies, in turn, are trying to figure out what kinds of AI they should invest in — because they don’t know what K-12 users will accept.

A recent set of guidelines published by the U.S. Department of Education seeks to help the two sides figure out next steps in AI product development.

The report, “Designing for Education with Artificial Intelligence: An Essential Guide for Developers,” highlights the importance of trust while creating AI products and recommends that developers make sure their work accomplishes the following:

  • Keep teaching and learning as the main priority by aligning their products with education-specific values and visions;
  • Show evidence of impact in their rationale behind creating products;
  • Work to close gaps in equity and bias, which have been a major concern with AI products;
  • Ensure protection of students and their data; and
  • Demonstrate transparency in product design and function and build trust with the school community.

“Trust improves the co-design process between developers and educators so that together they can create and scale innovative products,” the report says. “Because the technology is evolving rapidly, developers may find it valuable to go beyond attending to and complying with today’s federal guidelines and guardrails to earn trust.”

The guide was developed in response to President’s Biden’s October 2023 executive order directing the department to develop resources for safe, responsible, and nondiscriminatory uses of AI in education.

It also builds off the department’s May 2023 report on AI and the Future of Teaching and Learning.

Lindsay Jones, CEO at the education research and development nonprofit CAST, says one of the biggest concerns she hears among districts right now about artificial intelligence is that they don’t have a sense of who’s credible and reliable in maintaining student privacy, in helping kids learn, and in something that will support both students and educators.

Vendors looking to prove themselves to districts can look to the Department of Education’s AI guidelines as a design foundation to answer the questions that school leaders might ask them, she said.

“The market is really crowded right now,” she said, and districts need “ideas of how to know who to trust, what kind of questions to ask, and will [the technology] really increase student outcomes of learning.”

Some of the newly published guidelines reflect the Universal Design for Learning, said Jones, whose organization founded the framework that aims to make teaching and learning accessible for all.

“Having a tool like this as a developer, [which will] help you increase your market reach, is essential,” Jones said.

As school districts develop their own procurement guidelines for AI tools, they’ll be looking for any best practices they can find, and the AI guide will help, said Erin Mote, founder and CEO of InnovateEDU. Her nonprofit works to support equity in education and is part of the EDSAFE AI Alliance — one of the contributing organizations to the report.

Others who helped shape the guidelines include leaders from institutions and companies such as Khan AcademyGoGuardianCarnegie Learning, and the Worcester Polytechnic Institute.

This is the beginning of solution providers in the ed-tech space being able to educate their customers about their AI features that they’re using and building their own trust.

Erin Mote, CEO & Founder, InnovateEDU

Companies that are trying to gauge district appetites for AI and craft new products can help themselves by getting familiar with the kinds of principles established in the report early on, whether it’s incorporating those ideas into their development or procurement processes, Mote added.

“There’s still a giant AI literacy curve in our sector,” Mote said. “This is the beginning of solution providers in the ed-tech space being able to educate their customers about their AI features that they’re using and building their own trust.”

“That’s how we build deeper and better partnerships,” she said, “and sustain tools used for the long term in districts and in the state.”