AI Researcher François Chollet Launches Nonprofit for AGI Benchmarks
François Chollet is co-founding a nonprofit to create benchmarks for assessing artificial general intelligence
François Chollet, ARC Prize Foundation, AGI, AI Benchmarks, OpenAI, San Francisco, USA
San Francisco: So, François Chollet, the guy behind some cool AI stuff, is starting a nonprofit called the ARC Prize Foundation. The goal? To create benchmarks that really test AI’s ability to think like a human.
This new nonprofit will be led by Greg Kamradt, who used to work at Salesforce. They’re planning to kick off fundraising soon, which is pretty exciting.
Chollet mentioned that they want to be a guiding light for artificial general intelligence, or AGI for short. It’s a big deal because AGI means AI that can do most things humans can do.
They’re building on something called ARC-AGI, which is a test Chollet created to see if AI can learn new skills outside of what it was trained on. It’s like a puzzle where AI has to figure things out on its own.
He first introduced this test back in 2019. While many AI systems can ace tough math problems, the best AI could only solve about a third of the tasks in ARC-AGI until recently.
Chollet is clear that they’re not just trying to make AI take super hard tests. They want to close the gap between human and AI capabilities.
Last year, Chollet and another tech guy, Mike Knoop, started a competition to see if anyone could create an AI that could beat ARC-AGI. OpenAI’s new model was the first to get a decent score, but it took a ton of computing power.
Chollet has pointed out that ARC-AGI isn’t perfect. Some models can just brute-force their way to high scores, and he doesn’t think the new OpenAI model is truly human-level intelligent.
He believes that the next version of the benchmark will still be tough for AI, while regular humans will still score high without any training.
Knoop mentioned they’re planning to launch a new version of ARC-AGI soon, along with another competition. They’re also working on a third edition of the test.
There’s some debate about whether Chollet has oversold ARC-AGI as a way to reach AGI. The definition of AGI is still up for discussion, with some saying it’s already been achieved if AI can outperform most humans.
Interestingly, OpenAI’s CEO, Sam Altman, said they want to team up with the ARC-AGI folks to create future benchmarks. But Chollet didn’t share any updates on partnerships in his announcement.
On social media, the ARC Prize Foundation said they’re looking to build an academic network to push AGI progress and work with other AI labs on benchmarks.
And hey, if you’re into AI, TechCrunch has a newsletter that you can sign up for to get the latest news!
[rule_2]