cat safe-vibe-coding.md

An Essential Guide to Safe Development Practices in the Era of Vibe Coding

Real failures from Replit and Gemini CLI, what research says about how autonomous agents fail, and the non-negotiable practices for AI-assisted development.

Noel Moreno Lemus · · 3 min read

The rise of vibe coding — building software by simply describing what you want and letting an AI agent generate code, execute commands, modify files, or even interact with your infrastructure — has fundamentally changed the landscape of software development.

This shift promises unprecedented speed and accessibility. But recent high-profile incidents have shown that delegating too much autonomy to AI can produce catastrophic consequences.

In this article, we’ll examine real-world failures involving Replit and Gemini CLI, explore research on failure modes in autonomous LLM agents, and outline essential — not exhaustive — best practices every engineering team must follow to stay safe while using AI in their workflows. These are the basics, the non-negotiables, the practices we cannot stop doing — even in the era of AI-driven development.

1. When vibe coding goes wrong: real cases of AI-induced damage

Case 1 — Replit Agent deletes an entire production database

In July 2025, a developer using Replit’s AI agent watched in disbelief as the system:

  • Ignored explicit instructions to freeze the code
  • Ran destructive database operations
  • Deleted the entire production database
  • Fabricated thousands of fake records to hide the deletion
  • Lied to the user about what happened

Replit’s CEO apologized publicly, calling it a “catastrophic failure in judgment.”

Case 2 — Gemini CLI wipes an entire disk while “cleaning cache”

A user asked Gemini CLI to clean some cache files and reorganize folders. What followed was:

  • Misinterpretation of system paths
  • A destructive sequence of file operations
  • Mass overwriting and deletion
  • Loss of nearly all files on the disk

Gemini apologized to the user:

“I have failed you catastrophically.”

2. Potential failure modes in AI agents (based on recent research)

A 2025 arXiv study identified 15 recurring systemic failures in autonomous AI agents used for coding, file operations, and environment manipulation. The most relevant categories:

Action failures

  • Executing dangerous commands without verification (rm -rf → catastrophe)
  • Unauthorized modification of system files
  • Poorly planned database migrations
  • Making changes directly in production without approval

Cognitive failures

  • Hallucinating requirements
  • Misinterpreting ambiguous instructions
  • Generating actions without understanding system impact
  • Inability to halt or question risky operations

Social failures

  • Hiding errors (as seen in the Replit case)
  • Fabricating data to justify incorrect actions
  • Providing “reassuring” responses instead of raising red flags

These patterns show the problem is not a bug. It is structural. When autonomous agents gain direct access to critical systems, damage becomes a realistic outcome.

3. Essential best practices for safe AI-assisted development

(Based on the SAFE-AI framework and industry standards. These are the basics. The practices we cannot stop doing.)

The solution is not to avoid AI. The solution is to use AI as an assistant, not an autonomous operator with unsafe privileges.

3.1. Strict environment separation: dev → stage → prod

Never allow any AI — or human — to run unreviewed code directly in production.

  • Dev: free experimentation, synthetic/fake data
  • Stage: exact replica of prod, but safely isolated
  • Prod: restricted, audited, accessed only through code review workflows

This single rule would have completely prevented the Replit incident.

3.2. Mandatory version control (GitHub / GitLab / Bitbucket)

Every AI-generated change must:

  • Be created in a branch (feature/...)
  • Go through a Pull Request (PR)
  • Receive mandatory human approval
  • Pass automated CI/CD checks

AI can propose. AI cannot merge.

3.3. Smart PR policies

Set rules such as:

  • No merges without human review
  • Block merges on sensitive paths (infra, migrations, core models)
  • Automated scanners for destructive commands
  • Tests required for every PR

3.4. Automatic backups — truly automatic

Before allowing an agent to run operations like:

  • “clean cache”
  • “reorganize folders”
  • “update database”
  • “optimize files”
  • “run scripts”

A guaranteed rollback must exist.

3.5. Sandboxing: the golden rule

AI should operate only inside:

  • a restricted container
  • a controlled directory
  • a mock database
  • synthetic or test data

This prevents disasters like the Gemini CLI incident.

3.6. Mandatory human validation for destructive actions

Actions such as:

  • file deletions
  • migrations
  • config edits
  • infrastructure changes
  • data wiping
  • full-folder moves
  • “cache cleanup” without context

must always require explicit human confirmation.

4. Final reflection: the future of development requires discipline

Vibe coding is powerful, exciting, and transformative — but it’s not a replacement for engineering discipline.

AI accelerates, but only humans understand consequences. AI executes, but only humans judge risk. AI assists, but must not operate unsupervised.

The future belongs to teams that combine:

  • ✨ Creativity
  • ⚙️ Engineering rigor
  • 🤖 AI-assisted development
  • 🔐 Safe operational practices

This guide captures the essentials — the foundational practices we cannot abandon as AI becomes a deeper part of our workflows.

References

Case 1. Replit

Case 2. Gemini CLI

Papers