Chalkbeat Ideas is a new section featuring reported columns on the big ideas and debates shaping American schools. Sign up for the Ideas newsletter to follow our work.

Three years ago, as Khan Academy founder Sal Khan rolled out an AI-powered tutoring chatbot, he predicted a revolution in learning.

So far, the revolution hasn’t happened, he acknowledges.

“For a lot of students, it was a non-event,” Khan told me recently about his eponymous chatbot, Khanmigo. “They just didn’t use it much.”

Khan gives this analogy: Imagine he walked into a class, sat in the back of the room, and waited for students to seek out help. “Some will; most won’t,” he said. That’s been the experience with AI tutoring, he said. It doesn’t necessarily make students motivated to learn or fill in gaps in knowledge needed to ask questions.

Khan’s comments are an acknowledgement that AI has not quickly allowed for the creation of an effective super-tutor, as some initially hoped. It’s an early indication of the limits of AI to drive massive learning gains, long an unrealized goal of various technologies. While Khan remains optimistic about various uses of AI in education, he’s also come to see its limits.

“I just view it as part of the solution; I don’t view it as the end-all and be-all,” Khan said in our interview.

In the summer of 2022, OpenAI leaders Sam Altman and Greg Brockman reached out to Sal Khan. They were months away from releasing ChatGPT, and were hoping Khan Academy — a large nonprofit that works with schools across the country — could showcase the technology’s potential benefits. “I didn’t realize it yet, but the world was about to be turned upside down,” Khan wrote in his 2024 book “Brave New Words: How AI Will Revolutionize Education (and Why That’s a Good Thing).”

OpenAI provided Khan Academy with early access to a more advanced AI model, GPT-4. With that the Khan Academy team then built a specialized chatbot, Khanmigo, designed to help students learn and restricted from simply giving them the answer. Khan himself quickly became an evangelist for the technology’s uses in schools.

“We’re at the cusp of using AI for probably the biggest positive transformation that education has ever seen,” Khan said in a widely viewed TED Talk in 2023. “The way we’re going to do that is by giving every student on the planet an artificially intelligent but amazing personal tutor.”

He suggested that eventually AI could turn the average student into an academic standout, citing a seminal but controversial 1984 study on the value of individualized tutoring.

Khan also appeared in a “60 Minutes” segment that featured northwest Indiana’s Hobart High School, which was an early adopter of Khanmigo.

Kristen Musall, a geometry teacher at Hobart High, gave Khanmigo a try when it first rolled out. Musall appreciated its encouraging, teacher-like tone, but she found that students didn’t really care for the bot. They found it frustrating — Khanmigo sometimes made mistakes, but also wouldn’t give away the answer. “If students don’t engage with the material enough to know what they’re looking for, then an AI like Khanmigo doesn’t necessarily help,” she recently told me.

Musall no longer uses Khanmigo in her class. She says there’s been more enthusiasm for the product among administrators than teachers in her school.

A few of Musall’s most advanced students have taken advantage of AI to learn new topics. But, as far as she can tell, more students are using it to just find answers, which has created a massive headache for teachers. Nationally, a majority of teenagers say AI-powered cheating is at least somewhat prevalent in their schools, according to a recent Pew survey.

Peggy Buffington, Hobart’s superintendent, said there’s been a range of reactions from teachers and students to AI. There was initially a learning curve for students to ask Khanmigo questions, but that they’ve gotten a lot better, she said: “It’s like anything in education. You have to learn how to use the tool and use it appropriately.”

Buffington says that schools need to prepare students to use AI responsibly and Khanmigo is preferable to commercial products they would use on their own. Overall, she’s found the tool beneficial. “Our kids can log in at home and they can get help with their homework and it won’t give them the answer,” she said.

But Khan Academy officials have seen that many students won’t take advantage of that option or don’t know how to. Kristen DiCerbo, the organization’s chief learning officer, said AI can only respond to students based on what they ask. And it turns out, she said, “Students aren’t great at asking questions well.”

DiCerbo was initially hopeful that AI would be able to personalize instruction to students’ needs and interests. That hasn’t happened. “So far I am not seeing the revolution in education,” she said.

AI is still poised to shake up American education in many ways — by making cheating easier, reshaping how teachers approach their work, and changing the broader economy in ways that affect schools. So far the evidence base for AI in education remains “extremely limited,” according to an overview paper released last month.

Khan Academy officials say they’re learning from their experience with Khanmigo and pairing it with other offerings.

A recent study found that when teachers used Khan Academy to help students practice academic content, their classes made slightly faster learning gains. (Lower-performing students, though, saw few if any improvements from Khan Academy.) This was before Khanmigo.

Khan Academy recently announced an overhaul of its product that provides students additional academic practice. Now Khanmigo is incorporated directly as a way students can get advice as they’re working through specific problems. A spokesperson said the organization made this change because “students were not seeking out Khanmigo’s help as much as we had hoped.”

“AI is going to help,” said Khan of this reimagined Khan Academy. “But I think our biggest lever is really investing in the human systems.”

Matt Barnum is Chalkbeat’s ideas editor. Reach him at [email protected].