Thumbnail image for How I Evaluate New Technologies
August 30, 2026
engineering series
Complexity
Evaluation
Technology

How I Evaluate New Technologies

New technology doesn't earn a place in my stack by looking good on paper. It earns it by surviving a small, disposable trial first.

New technology usually gets judged by whether it’s good.

Is it well designed. Is it popular. Do people whose judgment I trust recommend it. If the answers are yes, that looks like enough reason to adopt it.

For a while, that was more or less my filter too.

Where That Filter Breaks Down

Kubernetes is the clearest case where it failed. On a small team I was part of, the decision looked defensible at the time. Microservices felt flexible. Kubernetes looked future-proof. If the system grew, the architecture would already be ready for it.

None of that was wrong on its own terms. It just answered the wrong question. We adopted infrastructure built for problems we did not have yet, on the assumption that we eventually would. We did not meaningfully need Kubernetes-level autoscaling, sophisticated scheduling, or advanced rollout machinery. What we got instead was expensive debugging, operational knowledge concentrated in too few people, and a growing amount of tooling whose whole job was managing the complexity we had just introduced.

The real cost was not the infrastructure. It was what the infrastructure displaced. Every hour spent maintaining a cluster was an hour not spent shipping product, and for a small team, product capacity is the scarcest resource there is. I wrote about that one in more detail in When Not to Use Kubernetes. Kubernetes had not earned its place. It skipped probation because it looked like the serious choice, and by the time that became visible, it was already load-bearing.

What Actually Works

What I do now instead is test anything new in a small, disposable context before it gets anywhere near something that matters. A side project, a throwaway script, a single container, something I can walk away from without cost. Only once it’s proven itself there does it get considered for the real stack.

Graduating out of that trial means clearing three questions, in order.

What real cost does it remove today, not hypothetically, not for a scale I don’t have yet. What happens when it turns out to be wrong, and can I still fix that myself, or does the failure land somewhere I can’t see it. Would I still want this once the novelty has worn off and it’s just another dependency I’m responsible for.

Kubernetes never really sat through that trial. It arrived pre-approved by reputation. Nothing forced the first question to get asked honestly, because the second and third never got asked at all.

What Passing Actually Looks Like

Devcontainers are the case that went the other way, and it’s a slower, less dramatic story precisely because it worked. They started as a small, disposable fix: isolating LaTeX dependencies for university assignments so they’d stop polluting my local machine. Nothing about that first use case demanded they become a default. They earned that gradually, by solving the same real problem again on the next project, and the one after that, until using them stopped being a decision and started being the default. That’s the shape a real graduation takes, documented as it happened in Why I Use Devcontainers and revisited a year later in Using Devcontainers Without Overcomplicating Your Setup, which is really a piece about the same restraint from the other direction: not adding to something just because you can.

The second question, about the cost of being wrong, got a sharper test recently with an AI coding assistant building an open-source Terraform provider. Most of what it wrote was correct, and the parts that weren’t tended to fail loudly: broken code that announces itself immediately. The one place that mattered more was quieter. A comment describing a quirk in a third-party system got written early, sounded plausible, and shipped unquestioned across two releases before a more pointed question forced an actual check against the source. The real behavior turned out narrower than what had been written down. A two-line fix, but the finding was upstream of the code entirely.

That’s the real distinction the second question is asking about. Loud failures are cheap, because they tell you immediately and point at themselves. Quiet, plausible-sounding wrongness is expensive, because nothing forces you to go check it, and it can sit in a system for months looking like a fact. I wrote about that trial in more detail in What Claude Code Got Right (and Wrong) Building a Terraform Provider. The tool passed. Not because it was always right, but because when it was wrong, the wrongness was still something I could catch and fix myself.

The Actual Filter

None of this is really about being skeptical of new things. It’s the same instinct behind deciding what not to build at all, described in How I Decide What Not to Build: adding is easy to justify and expensive to carry, so the bar for adding anything should be a real one, not a reputational one.

The filter was never whether something looks good. Plenty of things that look good never get past the first small, disposable trial. What actually decides it is whether it removes a cost I have today, whether being wrong about it is something I can still catch and fix myself, and whether I’d still want it once it’s no longer new.

If you're still here, might as well subscribe :)
Get notified via

Projects Featured In This Article

Related Articles