Elon Musk Says ‘If Its Predictions Are Not Very Good, It’s Not That Smart’ — The One Intelligence Test for Humans and AI

Sounding smart is easy. Calling the future correctly is the part nobody can bluff. Tesla (NASDAQ:TSLA) and SpaceX (NASDAQ:SPCX) CEO Elon Musk wants to grade intelligence on exactly that, for people and AI alike.

"Probably the right metric for intelligence is the ability to predict the future," Musk said at an America PAC town hall in Lancaster, Pennsylvania, in 2024.

"You’re as intelligent as you can predict the future well," he said. "Because if somebody claims this person or this AI is very intelligent, how good are its predictions?"

"If its predictions are not very good, it’s not that smart," Musk continued. "To predict the future, you have to think critically about the past and constantly try to be less wrong."

Confidence Is Cheap, Scorecards Are Not

Earlier in the town hall, Musk said new information should shift the odds on a conclusion rather than lock in how sure someone feels about it.

That matters most for AI, which can spit out a polished answer in seconds and still get it wrong. A convincing explanation and a correct one are two different things, and a forecast is one of the few places where the difference shows up on a scoreboard.

The Bots Are Showing Their Scorecards

Researchers are already keeping score. The Forecasting Research Institute runs ForecastBench, a benchmark that pits AI models against superforecasters, a small group of people known for consistently outpredicting everyone else.

According to the institute, AI systems have likely reached parity with those superforecasters. It said 17 submissions now rank above them on dataset questions on a preliminary leaderboard, and one AI system has outranked them on market questions for the first time.

Musk’s own xAI was in that race. In a January update, the institute said xAI’s Grok 4.20 Preview tied for second place behind the superforecasters. xAI, which Musk founded, is now a division of SpaceX called SpaceXAI.

Astral Codex Ten blogger Scott Alexander said all eyes at this year’s prediction market conference were on the AI superforecasters.

"I met an AI superforecaster startup founder who told me his AI had turned $35 into $2 million on Kalshi over seven months," Alexander wrote in a July blog post.

He wrote that claims from the startups themselves deserve skepticism, but that even a few top human superforecasters said they were no longer confident they could beat the bots. Humans still took the top two spots in the most recent seasonal tournament he described, with an AI in third.

Why The Victory Lap Can Wait

Not everyone is ready to hand over the trophy. Good Judgment Inc., a forecasting firm built around superforecasters, said ForecastBench compares AI against a human baseline last surveyed in 2024. It also said the benchmark’s data-heavy questions and multiple AI attempts could make an early AI win look bigger than it is.

A Metaculus review found that as of this past May, professional forecasters still beat individual bots.

Prediction also isn’t the whole story of intelligence. Creativity, learning and problem-solving count too, and one missed call doesn’t make a model dumb, just as one lucky guess doesn’t make an expert. Repeated results across different situations tell the real story.

That brings it back to Musk’s test. The machines are now being graded on their calls, and the humans haven’t been fully dethroned. His advice at the town hall still applies to both sides. Don’t aim to be right every time, aim to be less wrong.

Image: Shutterstock