Five days ago, Claude Code shipped a new command: '/security-review'. Run it inside a session and it reads back over your agent's own diff, hunting for the categories a human reviewer would flag in a pull request - injection, hardcoded secrets, missing or broken auth checks. It's on-demand. Nobody runs it for you; you type it, when you decide your agent's code is worth a second pair of eyes.
I wanted to know exactly what that review catches and, more importantly, what it doesn't. So I built a small app, planted two real bugs in it on purpose, and watched.
The setup: a Flask app with two bugs I put there myself
I'm not going to send you off to read someone else's writeup of someone else's bug. Build this with me. Here's the baseline - a Flask app backed by SQLite, one working, honest endpoint that reads an order by id with a parameterized query:
import sqlite3
from flask import Flask, g, jsonify, request, session
app = Flask(__name__)
app.secret_key = "dev-only-not-real"
DB = "orders.db"
def get_db():
if "db" not in g:
g.db = sqlite3.connect(DB)
g.db.row_factory = sqlite3.Row
return g.db
@app.route("/orders/<int:order_id>")
def get_order(order_id):
db = get_db()
order = db.execute(
"SELECT * FROM orders WHERE id = ?", (order_id,)
).fetchone()
if order is None:
return jsonify({"error": "not found"}), 404
return jsonify(dict(order))
That '?' placeholder is doing real work - the value never touches the SQL string itself. I committed that as the baseline, then wrote a follow-up change the way an agent actually writes one: two new endpoints for a feature request, "let people search their orders and cancel one." Here's what I added.
# search orders by item name
@app.route("/orders/search")
def search_orders():
q = request.args.get("item", "")
db = get_db()
rows = db.execute(
f"SELECT * FROM orders WHERE item LIKE '%{q}%'"
).fetchall()
return jsonify([dict(r) for r in rows])
# let a logged-in user cancel an order, and let household
# accounts manage shared orders too
@app.route("/orders/<int:order_id>/cancel", methods=["POST"])
def cancel_order(order_id):
if "user_id" not in session:
return jsonify({"error": "not logged in"}), 401
db = get_db()
order = db.execute(
"SELECT * FROM orders WHERE id = ?", (order_id,)
).fetchone()
if order is None:
return jsonify({"error": "not found"}), 404
is_owner = order["user_id"] == session["user_id"]
# household accounts get access via the shared_with table
shared = db.execute(
"SELECT 1 FROM shared_with WHERE order_id = ?", (order_id,)
).fetchone()
if not is_owner and not shared:
return jsonify({"error": "forbidden"}), 403
db.execute("UPDATE orders SET item = 'CANCELLED' WHERE id = ?", (order_id,))
db.commit()
return jsonify({"cancelled": order_id})
The first bug is the one you already know how to spot, because it's the classic: 'q' is a value the visitor typed, and it lands straight inside the SQL string through an f-string, no placeholder. I ran it against the real endpoint to prove it isn't theoretical:
GET /orders/search?item=nonexistent' UNION SELECT id, id, name, 1 FROM users -- [{"id":1,"item":1,"total":"sam","user_id":1}, {"id":2,"item":2,"total":"priya","user_id":1}, {"id":3,"item":3,"total":"iris","user_id":1}]
The 'orders' table has four columns - id, item, total, user_id - and a UNION has to hand back exactly that many values, in that order, or SQLite refuses to run it. I filled those four slots with id, id, name, 1 off the users table without lining them up to mean anything sensible in their new home, which is why the names surface in 'total' instead of 'item', and why 'user_id' comes back as a flat 1 three times instead of a real foreign key. Sloppy positioning on my part, not a weaker bug: the database still handed every name in the users table back through a field meant for grocery orders. Textbook SQL injection, and I mean that literally - it's the first example in every textbook.
The second bug is quieter, and it's the one this piece is actually about.
What the review catches
I ran '/security-review' on that diff myself, the same way you would on your own agent's work. Here's the finding it handed back, unedited:
# Vuln 1: SQL Injection: app.py, search_orders() * Severity: High * Description: User input from the 'item' query parameter is interpolated directly into a SQL string via an f-string and executed without parameterization, allowing arbitrary SQL injection into the orders table. * Exploit Scenario: A request to /orders/search?item=x' UNION SELECT id, id, name, 1 FROM users -- breaks out of the LIKE clause and reads arbitrary data from any table in the database, including user records this endpoint was never meant to expose. * Recommendation: Use a parameterized query - db.execute("SELECT * FROM orders WHERE item LIKE ?", (f"%{q}%",)) - matching the pattern already used by get_order() in the same file.
That's a correct, useful finding. It named the line, explained the exploit the way I'd just run it, pointed at the fix, and even noticed the older endpoint in the same file does it right. If this were the only bug in my diff, the review would have done its job completely and I'd have nothing else to write about.
It wasn't the only bug.
The one it didn't catch
Read 'cancel_order' again, specifically the ownership check. 'is_owner' is correct - it compares the order's own 'user_id' to whoever's logged in. Then there's a second check for the household-sharing feature: look up 'shared_with' by 'order_id', and if a row comes back, let the action through.
The 'shared_with' table has exactly two columns - 'order_id' and 'user_id' - because the whole point of the table is answering "shared with whom," not just "shared with someone." The query above only touches the first one: 'SELECT 1 FROM shared_with WHERE order_id = ?'. 'user_id' sits right there in the schema, and the cancel check never reads it. It only asks "has this order been shared with anyone, ever." Once one household account has legitimate access to an order, that single row makes the order fair game for every other logged-in account on the system, not just the one it was actually shared with.
I ran it. Sam has legitimate shared access to priya's order 101. Iris has none - never granted anything, not the owner, not shared with:
POST /orders/100/cancel (iris, order never shared with anyone) {"error":"forbidden"} POST /orders/101/cancel (iris, order shared with sam - not iris) {"cancelled":101}
Iris cancelled a stranger's order because someone else, entirely unrelated to her, once had a reason to touch it. That's a real authorization bypass, on a live server, and the review's report above doesn't mention it. Not as a caveat, not as a low-severity note. It isn't in there.
The fix is one line, and it's the same shape as the injection's fix: filter by the column that was sitting right there unused.
shared = db.execute( "SELECT 1 FROM shared_with WHERE order_id = ? AND user_id = ?", (order_id, session["user_id"]), ).fetchone()
I reran both requests against that version. Iris still gets forbidden on 100. She now gets forbidden on 101 too - and sam, the account 101 was actually shared with, still cancels it clean. One added column in a WHERE clause is the entire distance between a check that exists and a check that means something.
Why the review missed it
A clean report doesn't mean the model looked at this code and decided the logic was fine. It means it didn't recognize a pattern it's trained to flag - "is there a check before a write" is a pattern; "does that specific check filter by the right column" is a judgment call the review is tuned to stay quiet about unless it's certain. This one isn't missing - it reads like security code, an 'if not is_owner and not shared' guard sitting right where you'd expect one, checking a table that exists for exactly this purpose. Pattern-matching answers "did someone remember to add a permission check." It doesn't reliably answer "does this specific check enforce what it claims to," and telling those apart means understanding what "shared" is supposed to mean here, not just confirming a check exists - a scanner tuned to avoid crying wolf will let a plausible-looking check pass rather than flag something it can't be certain is wrong, which is exactly the class of bug you just watched it drop.
Why this matters this week specifically
"No findings" is a step you take, not a status your code carries around on its own: you type '/security-review', read what comes back, decide what "clean" means for this diff. That's worth being precise about, because it's tempting to read a clean report as "the logic is correct" when what it actually answered is narrower than that.
One question is "did this diff match a pattern the reviewer is trained to flag." The other is "is the logic correct." '/security-review' answers the first. It doesn't promise the second - and the bug two sections up is exactly the gap between them.
'/security-review' shipped five days ago. That's recent enough that most of you reading this haven't typed it yet, which is exactly why it's worth knowing what it's good at before you build the habit - not just that it exists.
Final thought
You don't need to read every line an agent hands you - that ship sailed the day these tools got good enough to trust with real work. But there's one category worth reading by hand every time, and now you know exactly which one: any check that decides who's allowed to touch, change, or delete something that isn't obviously theirs. Find the word 'session' or 'user_id' in your own code this week, follow it into whatever query sits underneath it, and check that the columns being compared are the ones that actually answer "does this belong to the person asking" - not just "does something exist." A clean report is a floor, not a ceiling, and this is the one gap it doesn't promise to cover.