Code Review Optimization
Last updated 2026-09-22What's new
- There's a debate in the software community about whether engineers should review their own code or rely on AI agents (AI tools that can write and review code) to do it, with some arguing that AI is already better at it.
- The speaker suggests a middle ground, advocating for a gradient approach to code review based on the importance of the code, comparing it to a tree where the trunk (core code) needs more attention than the leaves (less critical code).
- They recommend involving AI agents throughout the entire coding process, from planning to launching, and using feature gating (a technique to turn features on or off) to manage and test new features safely.
- The speaker also advises separating new code that doesn't interact with existing code (leaf nodes) from integration pieces, and using AI for simple validations on less critical code.
- Software factories use AI agents (computer programs that can perform tasks) to build software quickly and efficiently, like an assembly line.
- They're "model and harness agnostic," meaning they can work with any AI model or tool, focusing on workflow, skills, and domain knowledge.
- Software factories aim to maximize AI capabilities and speed up development, allowing you to ship high-quality software quickly.
- They involve creating a structured process, like isolating new features in separate branches to avoid conflicts, to streamline software development.
- Uber built a system called U Review (automated code review software) to speed up their software engineering teams' work, as manual code reviews were taking too long.
- U Review connects with Uber's existing tools, Fabricator (Uber's old code management system) and GitHub (a popular online code management service), and ensures consistent code review rules for both human engineers and AI agents (AI tools that help with coding tasks).
- The system collects feedback and data to improve its performance, tracking things like how often developers act on the AI's suggestions and the AI's decision-making process to reduce costs and increase quality.
- Uber found that AI models (the brains behind AI tools) need guidance and rules to work well, as they can be overly confident and may not catch team-specific coding styles or issues without proper training.
- Anthropic, an AI company, released a new tool called Fable, which helps users describe goals and lets the AI figure out how to achieve them, rather than breaking down tasks step-by-step.
- Mike Krieger, co-founder of Instagram, shared that he used AI to port a large Python codebase to TypeScript (a programming language) over a weekend, which would have been a massive task before.
- He also mentioned that AI tools are becoming more powerful, and people should be "unreasonable" in their usage, pushing the tools to do more complex tasks.
- Krieger suggested that AI products should have more freedom to access tools and environments, making them more useful for tasks like parsing PDFs or writing scripts.
- AI coding tools (like LLMs or large language models) are improving but still make small mistakes, so human engineers will still be needed to review and maintain AI-generated code.
- "Agentic coding workflows" (using AI tools to automate parts of coding) are evolving—balance trying new tools with sticking to what works for you.
- Spend time learning AI coding tools deeply (like reading all their guides) to uncover hidden features that can speed up your work.
- When reviewing AI-written code, focus first on safety (checking for errors) before looking at quality or cleanliness.
- Maven Clinic, a digital health platform, created Maven Intelligence, an AI system that helps employees and clients by automating tasks like meeting management and customer service.
- They integrated AI into their products to improve user experience and reduce costs, like using AI chatbots for 24/7 customer support.
- Maven Clinic encourages employees to use AI tools for tasks instead of delegating to others, and they hire people interested in AI and capable of solving complex problems independently.
- They reward employees who use AI to increase their impact and are changing their work processes to maximize AI benefits, like building and testing quickly with AI.
Key points
What it is
- Code review optimization uses AI tools to speed up and improve the process of checking code changes, preventing it from slowing down AI development.
- It focuses on aligning the team on what the code should do, not just finding bugs, and treats AI-generated code like code from a junior developer.
- The goal is to let AI handle most of the review work, then have humans step in to judge, iterate, and make final decisions.
How to use it
- Start by creating an AI slot register (a shared list of rules for your AI to follow) to maintain semantic accuracy (ensuring the code does what your team intends).
- Use AI tools to run review commands on new features, asking them to pressure-test tradeoffs, failure modes, and design choices.
- Make the review process iterative, using a loop to allow the AI and your coding assistant to go back and forth, catching more issues than a one-pass scan.
Watch out for
- Don’t treat code review as just a bug-catching exercise; focus on team alignment to avoid repeating the same issues.
- Don’t remove humans from the review loop too quickly; it takes months of infrastructure investment to trust AI review fully.
- Plan up front to reduce long, difficult reviews by aligning on key decisions early.
Tools named
- Codex (a tool for reviewing and generating code), Claude Code (an AI coding assistant)
Lesson 1: What is Code Review Optimization and why it matters
Code review optimization means using automated tools and processes to make the review of code changes faster, more accurate, and less of a bottleneck. In AI development, this matters because AI can generate code far faster than humans can review it. As one expert noted, if people can’t review code as quickly as AI generates it, human review becomes the bottleneck to AI development.
The core issue is alignment (getting everyone on the same page about what the code should do). One expert stressed that "code review is not just about code review, it is about getting the alignment." When AI writes code and AI reviews it, doing this in a traditional UI (user interface) becomes inefficient. Instead, you can use AI agents to talk to each other through multiple rounds of review until they both sign off on a plan.
A key idea is to plan up front to reduce the chance of a long, difficult review process. Treat AI output like code from a junior developer—review it carefully and never assume it’s correct. In fact, 48% of AI-generated code contains security vulnerabilities, so critical evaluation is essential. Some teams build security review systems to check AI skills for risks, treating them like supply chain dependencies.
The payoff is measurable: one engineering lead reported that code flowing through AI review, AI QA, and AI security analysis is 87% less likely to hit a bug. The goal is to let AI get you 90% of the way there, then you review, judge, and iterate quickly.
Sources
- 2026-08-17 — How to Kill the Code Review Ankit Jain, Aviator
- 2026-05-04 — Ralph Loops Build Dumb AI Loops That Ship Chris Parsons, Cherrypick
- 2026-07-20 — In the Land of AI Agents, the Verifiers Are King Tariq Shaukat, Sonar
- 2026-01-29 — From Coder to Orchestrator The Developer Role Shift Nobody's Talking About
- 2026-07-23 — Harness Engineering is not Enough Why Software Factories Fail Dex Horthy, HumanLayer
- 2026-06-05 — I Updated grill-me And Solved Claude Code
- 2026-06-05 — Its starting
- 2026-07-29 — We Vetted 2000 AI Skills Before They Reached Developers Lucas Palma, Nubank
- 2026-07-28 — How Forward Deployed Engineering is done at Factory Eno Reyes
- 2026-06-25 — I asked Claude Code to make me as much money as possible
Lesson 2: How to use Code Review Optimization: step-by-step
To kill code review (the painful backlog of manual checks), you start by aligning the team, not just fixing bugs. First, create an AI slot register (a shared list of rules your AI must follow) to keep semantic accuracy (meaning the code does what your team intends). This registry takes time to build—expect a J curve where initial effort is high before payoff.
Next, run a review command on your new feature. For example, use Codex’s `codex review` or an adversarial review (a read-only check that questions your implementation). This review is steerable, so you can ask it to pressure-test tradeoffs, failure modes, or whether a simpler approach works. Unlike basic bug hunting, it examines design choices too. Run it alongside computational review (automated static analysis) and LLM-driven review (AI reading for logic issues) because no single method catches everything.
To optimize, make the review iterative. Use a loop like `{slash}loop 5 min` so the AI and your coding assistant go back and forth up to five turns, catching more issues than a one-pass scan. Wire this into pull request (PR) review—ask the AI to report every finding, then filter only what matters. Keep humans in the loop initially; removing them fully takes months of infrastructure investment. Always preserve evidence of what the AI reviewed before you optimize for speed.
Example: run `codex review` on your plugin, then loop in a second AI to adversarially question its logic. Fix, re-run, and merge when the review stops surfacing real problems.
Sources
- 2026-08-09 — Guide, Verify, Solve Anirban Chatterjee, Sonar
- 2026-08-17 — How to Kill the Code Review Ankit Jain, Aviator
- 2026-07-23 — Harness Engineering is not Enough Why Software Factories Fail Dex Horthy, HumanLayer
- 2026-03-31 — This Plugin Makes Claude Code 50x Better At Coding
- 2026-04-08 — The Next Layer After Prompt Engineering — Archon V3 Explained! 🚀
- 2026-07-15 — Simon Willison in conversation with Cat Wu & Thariq Shihipar, Anthropic
- 2026-06-05 — I Updated grill-me And Solved Claude Code
- 2026-03-31 — Codex Just 10x’d Claude Code Projects
- 2026-08-01 — Zero to Claude Certified Architect Professional Part 16 Team Enablement with Claude Code
- 2025-12-15 — n8n's New Chat Hub Release What You Need to Know
- 2026-07-22 — Prompting is dead. Here is how you create loops
- 2026-05-06 — Master 97% of Codex in 1 Hour (full course)
Lesson 3: Best practices and pitfalls
Code review optimization pitfalls and mistakes
The biggest pitfall is treating code review as a pure bug-catching exercise. As Ankit Jain notes, code review is not just about code review — it is about getting alignment across the team. When you skip this alignment, you end up re-identifying the same issues repeatedly. The fix is to build an "AI slop registry" (a stored list of common errors) so recurring feedback gets codified instead of re-typed each review.
Another mistake is removing humans from the review loop too fast. Simon Willison's conversation stresses that trusting AI review requires months of investment in infrastructure to give confidence it catches everything you care about. Remove people gradually, not overnight, or you regress on safety.
Move the review point earlier. Matt Dailey and Dex Horthy both emphasize planning up front to reduce long, difficult reviews. Once you align on key decisions early, the code review itself becomes easier because the hardest part—design tradeoffs—is settled before code exists.
Use iterative, adversarial review. The Codex plugin example shows an AI reviewer that questions chosen implementation and design, pressures tests tradeoffs, and checks failure modes. This goes beyond standard review-on-steroids because it is steerable and read-only, giving an audit without changing code.
Finally, structure your workflow. In Claude Code, keep context first, then implementation, then tests, so review discovers drift before it becomes costly. Run parallel reviewers for complex PRs, but skip full review overhead for light features.
Sources
- 2026-08-17 — How to Kill the Code Review Ankit Jain, Aviator
- 2026-07-15 — Simon Willison in conversation with Cat Wu & Thariq Shihipar, Anthropic
- 2026-08-09 — Velocity Sickness What Happens When Your Whole Team Gets 10x Faster Matt Dailey, Ref.
- 2026-07-23 — Harness Engineering is not Enough Why Software Factories Fail Dex Horthy, HumanLayer
- 2026-06-05 — I Updated grill-me And Solved Claude Code
- 2026-03-31 — Codex Just 10x’d Claude Code Projects
- 2026-08-01 — Zero to Claude Certified Architect Professional Part 16 Team Enablement with Claude Code
- 2026-07-17 — The Great Loops Debate Dex Horthy, Geoff Huntley, Ian Livingstone, Greg Pstrucha, insecure-agents
- 2026-04-08 — The Next Layer After Prompt Engineering — Archon V3 Explained! 🚀