Empowering Instructors | Supporting Excellence
Are You Smarter Than AI?
By RACHAEL J. WEBB and MAGGIE GRADY
This assignment is intended to teach students the strengths and limitations of Generative AI tools, and the value in struggling to solve a problem.
In 2019, software engineer and AI researcher François Chollet published his seminal paper “On the measure of intelligence." In this paper, he introducd the world to his "Abstraction and Reasoning Corpus for Artificial General Intelligence,” or “ARC-AGI” for short.
Artifical General Intelligence, or AGI, is artificial intelligence that aims to duplicate human intellectual abilities (Copeland, 2026). AGI has long been the finish line in the global competition to produce the most robust aritifical intelligence program (Hao, 2025). OpenAI, Anthropic, Google, and others are racing to achieve AGI before their competitors. But how do we know we have achieved AGI? The ARC-AGI is Chollet’s attempt to answer that question: it is a test of some 400 task-based items (Chollet, et al., 2026).
The ARC-AGI is an attempt to “create a fair and meaningful comparison between artificial intelligence and human intelligence” (ARC Prize, 2026). The test deliberately restricts itself to problem-solving tasks that are innate in humans, or learned at a very young age (see, for example, Spelke & Kinzler’s (2007) research on Core Knowledge theory). The items in the assessment are meant to be easy for humans to figure out quickly with no prior instruction.
Software developers are tasked with developing an AI system that preforms as well as humans on the ARC-AGI, and prize money is awarded to the top development team.
All students are expected to engage with the ARC-AGI-3 instrument and reflect upon the experience.
Navigate to https://arcprize.org/arc-agi and read the short overview of the ARC-AGI Series. You are also encouraged to watch the 3-minute video of François Chollet explaining his motivation for creating the ARC-AGI. Take notes as you read.
Once familiar with the corpus’s orgins, navigate to https://arcprize.org/tasks. Here, you will see a small thumbnail image of the task and its associated Task ID. Scroll through and select a test that appeals to you. For example, as someone who is not a “gamer,” I was drawn to re86. Each test has approximately eight levels. Begin your selected test by clicking on its Task ID.
After clicking “Start,” take periodic screenshots of your test to document your progress and reference later when you reflect upon this experience.
Students are encouraged to progress through all eight levels of their selected Task ID, as the tasks become more challenging with each level. However, you are welcome to explore other Task IDs, as well. Spend a minimum of 20 minutes engaging with the ARC-AGI puzzles to truly understand how they operate.
Design one, compelling, visual slide, incorporating the following:
- Reflect on your experience engaging with the ARC-AGI-3.
- Include images of the tasks you completed and detail mistakes you made when figuring out the task.
- Respond to Fei-Fei Li’s arguments that spatial intelligence must be achieved before AGI can be realized.
- Do you agree with Dr. Li? Why or why not?
- In her work, Li (2025) identifies several careers that are founded on having innate spatial intelligence. Explain how spatial intelligence- or a lack thereof- would impact your discipline.
UB faculty, staff and students may choose to utilize the University at Buffalo’s branded PowerPoint template to design your slide. External users are encouraged to utilize their home institution's templates.
ARC Prize. (2026). ARC-AGI series: Benchmarks for general intelligence. Retrieved from arcprize.org: https://arcprize.org/arc-agi
Chollet, F., Knoop, M., Kamradt, G., Wexler, D., Smith, D., Henry, H., . . . Cruz, M. (2026, March). ARC prize 2026 - ARC-AGI-3. Retrieved from kaggle.com: https://www.kaggle.com/competitions/arc-prize-2026-arc-agi-3/overview
Copeland, B. J. (2026, March 26). Is artificial general intelligence (AGI) possible? Retrieved from Britannica.com: https://www.britannica.com/technology/artificial-intelligence/Is-artificial-general-intelligence-AGI-possible
Hao, K. (2025). Empire of AI: Dreams and nightmares in Sam Altman's OpenAI. Penguin Press.
Li, F.-F. (2025, November 10). From words to worlds: Spatial intelligence is AI's next frontier. Retrieved from Dr. Fei-Fei Li: https://drfeifei.substack.com/p/from-words-to-worlds-spatial-intelligence