Using AI for System Design: Where It Helps, Where It Hurts

System design is the highest-leverage work I do. I wanted to know where AI could extend my leverage and where it would quietly degrade it. Eleven months of deliberate experiments gave me a clear answer.

Where AI Helped

Generating option trees: given a problem, list five viable architectures. The model is excellent at breadth. I am the one picking.

Stress-testing my diagrams: paste the architecture, ask what fails under load, list failure modes. The model catches single points of failure I miss.

Drafting the ADR: I write the decision, the model drafts the context and consequences sections. Saves 30 minutes per ADR.

Explaining my design to juniors: I describe, the model rephrases for the audience. Translation, not design.

Where AI Hurt

Picking between two good options: the model has no skin in the game, no knowledge of my clients, no cost profile. Its answer is plausible and wrong often.

Estimating capacity: the model cheerfully multiplies numbers. Real estimates require real traffic data from similar systems. Do not outsource this.

Choosing defaults: the model copies the most common default from training data, which is often outdated. Check the actual docs.

The Failure Mode I Hit Twice

I asked Claude to design a queue-based pipeline and accepted the first answer. It suggested SQS plus Lambda plus DynamoDB. Reasonable on paper. Wrong for my workload which needed ordered processing and ended up costing a full rewrite. The lesson: AI is excellent at producing reasonable. It is weak at producing right.

The New Process

1. Human writes the problem and constraints
2. AI generates five candidate architectures
3. Human eliminates three using context AI cannot have
4. AI stress-tests the remaining two
5. Human picks, AI drafts the ADR

This process beats pure-human and pure-AI design. It preserves judgment where judgment matters and saves time where time is cheap.

Recommended Reading

AWS Well-Architected Framework is still the reference I start from. AI is fast. The framework is right.