How to Answer
“I’d sort the system into two piles: things that get more valuable as models improve, and scaffolding that exists only because the current model is limited.
The durable pile: evals, because a more capable system needs them more, not less. Data and retrieval — proprietary context is the moat, and no model gets it for free. Tool surface and integrations, because a smarter model still cannot call an API you never built. Auth, permissions and audit, because those are about who is allowed, and no model improvement changes that. And the product decisions: what should happen, and what a good outcome even is.
The scaffolding pile: chaining a task into six steps because the model can’t hold it in one. Aggressive context compression. Routing between three model sizes. A fine-tune that exists to fix a formatting problem. Those are compensations, and every one of them is work I will eventually delete.
So I’d build the scaffolding when I need it, keep it thin, and keep it behind an interface so deleting it is a small change. What I would not do is architect the whole system around a limitation with a twelve-month half-life.”