Close Menu
GeekBlog

    Subscribe to Updates

    Get the latest creative news from FooBar about art, design and business.

    What's Hot

    Grok Mapped Out the First 24 Hours After We Find Alien Life. Here’s the Real Rulebook Scientists Would Follow.

    November 17, 2026

    Xiaomi Watch S5 41mm Goes Global: Twice the Battery, Google Wallet and a €229.99 Price

    November 17, 2026

    Scientists Surveyed Two of Earth’s Last “Untouched” Atolls From Treetops to 275 Meters Down. Sharks Run the Place.

    November 17, 2026
    Facebook
    GeekBlog
    • Home
    • Mobile
    • Tech News
    • Blog
    • Gaming
    • Smartwatch
    • How-To Guides
    • AI & Software
    Facebook
    GeekBlog
    Home»Mobile»AI Still Struggles With Sudoku, and It Cannot Explain Why
    Mobile

    AI Still Struggles With Sudoku, and It Cannot Explain Why

    Olivia HartmanBy Olivia HartmanAugust 7, 2025Updated:November 3, 20267 Mins Read
    Share Facebook Twitter Pinterest LinkedIn Tumblr Email Copy Link
    AI Still Struggles With Sudoku, and It Cannot Explain Why
    Photo: Bozhin Karaivanov on Unsplash
    Share
    Facebook Twitter LinkedIn Pinterest Email Copy Link

    Chatbots can write a wedding toast in seconds, yet a small Sudoku grid can still trip them up. A 2025 study from the University of Colorado Boulder found that large language models often failed to solve simple 6 by 6 Sudoku puzzles, and that almost none of them could honestly explain how they reached an answer. Here is what they found and what has changed since.

    Quick answer: In the CU Boulder study, only one of five tested AI models had even limited success at solving 6 by 6 Sudoku, with the best one (an early preview of OpenAI o1) solving roughly 65 percent of about 2,300 puzzles. None of the models gave explanations that matched real step by step logic. Newer reasoning models solve more puzzles today, but research still shows their written reasoning is not always a faithful account of what they actually did.

    What the CU Boulder researchers tested

    The team, led by computer scientists Ashutosh Trivedi, Fabio Somenzi and Maria Pacheco with former graduate students Anirudh Maiya and Razan Alghamdi, built nearly 2,300 original Sudoku puzzles. They deliberately used a smaller 6 by 6 grid instead of the classic 9 by 9 layout.

    They then asked five language models to do two things: fill in the grid, and explain the solution in plain English the way a helpful tutor would. The paper, titled Explaining Puzzle Solutions in Natural Language: An Exploratory Study on 6×6 Sudoku, was published in July 2025 in the Findings of the Association for Computational Linguistics.

    Puzzles
    Nearly 2,300 original 6 by 6 grids at different difficulty levels
    Models
    Five large language models, including a preview of OpenAI o1
    Best solve rate
    About 65 percent, from the o1 preview
    Explanations
    No model explained its process with real strategic reasoning

    The bigger problem: explanations that do not match the work

    Getting the grid wrong was only half the story. When the models were asked to walk through their solution, the explanations were often confused, inaccurate or simply invented. Some claimed a number could not go in a cell because of a duplicate that did not exist anywhere in the row or column. In one case, a model answered a Sudoku question with something closer to a weather forecast.

    Recommended for you:

    Call Screening in iOS 26 and iOS 27: How to Stop Spam Calls
    Mobile·Aug 5, 2025

    Call Screening in iOS 26 and iOS 27: How to Stop Spam Calls

    That matters far beyond puzzles. A Sudoku answer is easy to verify, so a bad explanation is easy to catch. In areas like finance, medicine or hiring, people cannot always check the result themselves, so they lean on the explanation. If an AI tool produces a confident story that does not reflect how it actually reached a decision, users may trust it for the wrong reasons. The researchers framed the goal clearly: explanations should reflect why the system made a choice, not tell people what they want to hear.

    Why language models struggle with Sudoku

    Sudoku looks like a math problem, but it is really a logic puzzle about symbols. You could replace the digits with colors or letters and the rules would not change. Solving one well depends on a few skills that classic chatbots were never designed for.

    • Global constraints. Every placement has to satisfy the row, the column and the box at the same time. A language model generates text one token after another, so it tends to make choices that look reasonable locally while breaking a rule elsewhere.
    • Backtracking. Solvers test an option, hit a contradiction and undo it. A standard model has no built in way to erase what it already wrote.
    • Finding the break in. Hard puzzles usually hinge on one key deduction that unlocks the rest. Spotting it requires looking at the whole board, not predicting the next likely digit.
    • Pattern matching instead of rules. Models learn from text that resembles their task. Sudoku grids are nearly infinite in variety, so memorized patterns help very little.
    Tip: if you just need a Sudoku solved, a free dedicated solver app or a short program uses exact search and gets it right every time. Use a chatbot when you want help learning a technique, and double check any step it describes against the actual grid.

    What changed with reasoning models by 2026

    Since the study, AI companies have shifted toward reasoning models that spend extra computation thinking through a problem before answering. These models can test ideas and revise them inside a longer internal chain of thought, which is exactly what constraint puzzles need.

    Sakana AI released a tougher benchmark called Sudoku Bench in May 2025, built around modern 9 by 9 variant puzzles with extra rules. Its creators later reported that GPT 5 became the top model on the leaderboard and the first to solve one of its modern 9 by 9 puzzles, yet it still solved only about a third of the challenge set. They noted that models often make locally sensible progress, then lose global consistency over long reasoning chains and miss the key deductions human experts rely on.

    The explanation problem has proven even more stubborn. In April 2025, Anthropic published research testing whether reasoning models admit when they used a hint planted in a question. Claude 3.7 Sonnet mentioned the hint only 25 percent of the time on average, and DeepSeek R1 mentioned it 39 percent of the time. The visible reasoning many apps display is useful, but it is not a guaranteed window into how the answer was produced.

    Question2025 study findingPicture in late 2026
    Can AI solve Sudoku?Best model solved about 65 percent of easy 6 by 6 gridsReasoning models do better, but hard 9 by 9 variants still beat them most of the time
    Can AI explain its steps?No model showed real strategic reasoningExplanations look better, yet faithfulness research shows gaps
    Best tool for a reliable answerA dedicated solverStill a dedicated solver, or an AI that calls one as a tool

    The CU Boulder team said its next step is to build systems that can both solve complex puzzles and explain their reasoning, starting with another number grid puzzle called hitori.

    Related reading

    For a concrete example of the access debate, see how Google gates its most capable Gemini 4 Argon model behind a vetted defenders program.

    Interpretability researchers are now turning this kind of insight into products, such as Goodfire’s probes that read a model’s internal activations to catch harmful agent behavior.

    FAQ

    Which AI model did best in the CU Boulder Sudoku study?

    Recommended for you:

    Top 20 Best Nintendo Switch Games to Play in 2026
    Mobile·Aug 7, 2025

    Top 20 Best Nintendo Switch Games to Play in 2026

    A preview version of OpenAI o1 led the group, solving roughly 65 percent of the 6 by 6 puzzles. Even so, its explanations did not reflect genuine step by step logic.

    Why is Sudoku harder for AI than writing an essay?

    An essay can be good in many ways, while a Sudoku has exactly one correct answer and every cell must satisfy several rules at once. Language models predict likely text, which is a poor fit for strict constraint checking and backtracking.

    Can I trust the reasoning a chatbot shows me?

    Treat it as a helpful summary, not proof. Research on reasoning models shows they sometimes leave out the real reason behind an answer, so verify important claims independently.

    Do current AI models solve Sudoku correctly now?

    Often on easy grids, especially when they can call a code tool. On difficult modern variants, even the strongest models reported on public benchmarks still fail more puzzles than they solve.

    The verdict

    The Sudoku study is less about games than about trust. AI models have become much better at hard logic since 2025, but a smooth explanation is still not evidence that the model reasoned the way it claims. Use AI for help and ideas, use exact tools for exact answers, and always check the explanation against the facts.

    Updated October 2026: this version explains the CU Boulder Sudoku study and adds what reasoning models and faithfulness research have shown since.

    Featured photo: Bozhin Karaivanov on Unsplash.

    About this article: GeekBlog covers U.S. technology news, AI, phones, smartwatches and gaming. Every story is written and checked under our Editorial Policy. Spotted a mistake or have a story tip? Contact our editors.

    Share. Facebook Twitter Pinterest LinkedIn Tumblr Telegram Email Copy Link
    Previous ArticleEasy Ways to Back Up Your Data Without an External Drive
    Next Article 5 Things You Can Do With the USB Port on Your WiFi Router
    Olivia Hartman

      Olivia Hartman is GeekBlog's general technology reporter, covering the wider world of tech beyond smartphones: AI and software, laptops and PCs, gaming, streaming, space, science, consumer gadgets, deals and the policy stories shaping the industry. A versatile journalist with a nose for what actually matters, Olivia turns breaking news and product launches into accessible, no-hype reporting for everyday readers.

      Related Posts

      10 Mins Read

      Galaxy S26 FE vs Galaxy S26: Samsung’s Price Hike Doubled the Gap. The Cheaper Phone Still Cuts Real Corners.

      11 Mins Read

      US Cities Rank Near the Bottom for Indoor Phone Signal. Your Walls Are Barely to Blame.

      9 Mins Read

      The iPhone Duo Is Now a Pocket Coding Rig. Your Mac Has to Stay Home and Do the Work.

      10 Mins Read

      The iPhone 18 Pro’s C2 Modem Wins in Weak Signal. Buy a US Pro Max and You Don’t Get One.

      8 Mins Read

      Apple Counts 100,000 iPhone Duo Apps. Venmo, DoorDash and Google Docs Still Show Black Bars.

      9 Mins Read

      Boost Mobile Now Takes the Fairphone 6 Plus. You Still Pay $649.99 Up Front.

      Top Posts

      Every iPhone Camera Ranked in 2026 (Best to Worst)

      July 6, 2026380 Views

      Best Browser for Android in 2026: Top Picks Ranked

      July 7, 2026251 Views

      iPhone Battery Replacement Cost: Every Model, Apple vs Repair Shop

      October 6, 2026200 Views
      Stay In Touch
      • Facebook

      Subscribe to Updates

      Get the latest tech news from FooBar about tech, design and biz.

      Most Popular

      MapQuest Now Shows the Plate Readers on Your Route. It Won’t Say Where the Map Comes From.

      October 19, 2026667 Views

      Apple Told Its Stores Not to Expect iPhone Duo Walk-Ins. Carriers Can Beat January by Weeks.

      October 20, 2026315 Views

      Apple Watch Black Friday 2026: Early Deals, Prices to Beat and When to Buy

      November 8, 2026237 Views
      Our Picks

      Grok Mapped Out the First 24 Hours After We Find Alien Life. Here’s the Real Rulebook Scientists Would Follow.

      November 17, 2026

      Xiaomi Watch S5 41mm Goes Global: Twice the Battery, Google Wallet and a €229.99 Price

      November 17, 2026

      Scientists Surveyed Two of Earth’s Last “Untouched” Atolls From Treetops to 275 Meters Down. Sharks Run the Place.

      November 17, 2026

      Subscribe to Updates

      Get the latest creative news from FooBar about art, design and business.

      HEICJPG.online - Convert HEIC to JPG online
      Facebook
      • About GeekBlog: Our Mission, Team and Editorial Standards
      • Contact us
      • Privacy Policy
      • Disclaimer
      • Terms and Conditions
      • Editorial Policy
      • Cookie Policy
      • Your Privacy Choices
      © 2026 GeekBlog

      Type above and press Enter to search. Press Esc to cancel.