Code audit

AI-Coded Apps Nail the Hard Part and Miss the Dumb One

8 min

Auditing an AI-coded app: the hard part nailed, the dumb part missed

The context

I audit apps coded with AI. Real products, with real users, built fast and working. Backend, frontend, database: code largely generated by an AI. The kind of project you can stand up in a few weeks today.

Each time, I run them through the same angles: security, architecture, quality, deployment. And each time, the same finding comes back, the one that still surprises a lot of founders.

You'd expect the usual cliché: "AI codes badly, holes everywhere." It's the opposite. The genuinely hard things to get right are often done right. And the simple things, the ones no tutorial reminds you about, are missing.

The most serious risk, the only truly irreversible one, is often the cheapest to fix. A few minutes of work. And it's almost always the one nobody thought to do.

Let's look at why, because this pattern shows up on almost every vibe-coded app I touch.

What the AI nailed (and it's not obvious)

Let's start with the good, because it's real.

When I open an app for a security audit, there's a handful of things I check first. The "dangerous to miss" things, the ones that cost a fortune to fix after the fact. And most of the time, they're clean:

  • Passwords are hashed properly. Not stored in plain text, not with some old broken algorithm. With bcrypt, and sometimes even an extra protection against a timing attack (I'll spare you the details, but it's above what a lot of teams do).
  • Access is re-checked on the server. Every time a user asks for some data, the server re-verifies that it actually belongs to them. No "I trust the URL." This is THE classic trap, and it's often avoided.
  • The crypto is sound. No fake algorithm, no dangerous shortcut.
  • A real test suite, green on the critical paths.

If you don't write code, just hold on to this: the core of the safe is well built. The lock, the hinges, the armored door: solid work. That's exactly what's hardest to fix if you get it wrong at the start.

Why does AI nail this? Because these are heavily documented problems. "How to hash a password" has ten thousand good examples online, and the AI has read them all. On well-charted topics, it's strong.

What the AI missed (and it's always the same)

The problem isn't the code. It's everything around the code.

An app running in production isn't just a program. It's a program plus an environment: a server, a config, access rules, backups, updates. And that, the AI doesn't generate. Nobody asks it to. It doesn't show up in the demo.

Here's what I find, and it's almost always the same list:

An admin password written straight into the code. The most powerful account in the app, the one that can do anything, has its password spelled out in black and white in the files. Often even in the project history, so impossible to truly erase. Anyone who gets hold of the code takes full control. Fix: pull the password out of the code, rotate it. Ten minutes.

No protection against mass login attempts. An attacker can try passwords endlessly, at full speed, never getting blocked. It's like an armored door... where you can try the codes one by one, as many times as you want, without anyone noticing. Fix: lock out after a few failures. Half a day.

Dependencies never updated. A modern app is your code plus dozens of ready-made bricks written by others. Several of those bricks carry known, published flaws. "Known and published" means attackers already have the manual. Nobody has run the updates. Fix: half a day.

The containers run with full privileges. Technical detail, but the idea is simple: if the app gets compromised, the attacker doesn't stay stuck inside the app, they walk straight out onto the whole server. Two lines of config to change.

Notice the common thread? None of these things are hard. They're not subtle bugs. They're omissions. Boxes never ticked, because nobody knew they existed.

The worst one is the simplest

And then, the finding that always makes me pause the most.

Very often, all the data (the users' work, the database, everything) lives on a single server. One machine. No copy anywhere else. Nowhere.

No backup. Zero.

Take a second to measure what that means. A hard drive that dies (it happens). A wrong move (happens even more). Ransomware that encrypts everything (happens to the best). And then it's over. Not "over, we'll fix it in two days." Over permanently. The data doesn't come back.

That's the huge difference with every other problem. A security flaw, you fix it, and it's done. Lost data, you don't "fix." It's gone.

And the craziest part: it's the cheapest thing to fix on the whole list. Setting up an automatic encrypted backup, going off to another server every night, and testing it once to make sure it works, is a few hours of work. It's the cheapest insurance in the world.

And nobody thinks of it. Not out of negligence. Because a backup isn't part of the code. The AI builds the app. It doesn't build the safety net underneath. And when you vibe code while watching your app run on screen, you have no reason to think about the day the server catches fire.

It's exactly the house analogy: you've got a top-of-the-line smart lock, an armored door, cameras. And no roof. As long as the weather's nice, everything's fine.

Why it's always this pattern

It's no accident. The AI optimizes for one thing: that it works now, on screen. Anything that doesn't show up in the demo (a backup, an up-to-date dependency, what happens if the disk dies on a Sunday night) slips through the cracks. Not out of incompetence: because nobody asks it to.

Those reflexes don't come from a tutorial. They come from having already hit the wall. The AI has never hit the wall. It generates code that works, not code that has already suffered.

That's exactly the job a human has to pick back up. Not rewrite the app, it's often good. Close the doors nobody saw.

The checklist before you go live

If you're vibe coding an app and about to put it in the hands of real users, here are the boxes the AI almost certainly didn't tick for you. None of them are complicated.

  1. An automatic backup, going somewhere other than your main server, that you've tested at least once. If you do only one, do this one. A backup that's never been tested is a backup that doesn't exist.
  2. No password, no key, no secret written into the code. All of that lives in "environment variables," kept separate from the code. If it's in the code, assume it's public.
  3. A limit on login attempts. After a few failed tries, you block. Otherwise your armored door gets its codes tried forever.
  4. Up-to-date dependencies. Those ready-made bricks need refreshing regularly. An attacker hunting for a known flaw is the lowest effort there is for them.
  5. A look at the server's permissions. The app shouldn't run with full powers over the machine. If something goes wrong, that limits the damage.

You'll notice none of these boxes say "rewrite your app." They're hours of work, not weeks. That's the good news about well-built vibe-coded apps: the structure holds, you just need to put the roof on.

The problem is never the AI

The problem is believing that "it works on screen" means "it's ready for real life." An app isn't just code that runs on demo day. It's code that survives a Sunday night, a wrong move, a dying disk. And for now, the human still matters.


Want to know what's hiding in your app?

If you've vibe coded a product and you're about to ship it to production (or it's already there), I can do the same kind of pass: I read the code and the config, list the real risks ranked by severity, and give you a prioritized fix plan. Not theory, concrete actions. Check out the Audit offer.

And if you want more audit takeaways in this style, sign up for the newsletter. Just the traps worth avoiding, no spam.

Sébastien Vanson

Sébastien Vanson

Software engineer with 11+ years of experience. I help founders building with AI go from prototype to production-ready product.

Newsletter

Stay in the loop

Practical tips on shipping AI-built products to production.
No spam, unsubscribe anytime.