Revisiting Self-Hosted AI

Last Saturday I visited Erik de Bruijn from Stekker, and I have the feeling I caught a glimpse of the future. I saw a computer so cool that I simply HAD to take a selfie with it.

I’ve always been a bit skeptical about running AI models locally. I tried it about a year ago. It was slow, cumbersome, and the output was honestly mostly nonsense. Fun to play around with, but not something you’d want to build anything serious on.

That picture no longer holds true, as I learned last Saturday.

Today’s open source models are truly on another level. Together with Erik, I ran OpenAI’s GPT-oss and Whisper models locally, and the results were insanely good.

The speed was perhaps the most impressive part. Erik asked a question from his laptop to the model. I heard the server’s power supply quietly crackle, the fans spun up briefly, and within 500 milliseconds the answer was already there. This is genuinely much faster than ChatGPT or Claude Code. It felt truly futuristic. Especially when you’re standing next to a machine with two NVIDIA RTX 6000 cards inside, and you realize this is all happening locally. I could literally hear the computing power at work. I’m used to being able to grab a coffee while an agent runs, but maybe that won’t be necessary anymore soon.

Running models locally obviously has clear downsides too. It’s expensive. The hardware is hard to get. It’s not plug-and-play. But the advantages are real. You’re the only user. Your data stays local and therefore secure. And you don’t need internet. Plus: the performance is insane.

I notice I need to revise my earlier assumptions about this. What once felt hobbyist and inefficient is now starting to become seriously production-ready. I think WBSO.ai could really benefit from this. If we can run certain features through a local model, we’d have some truly beautiful advantages.

Self-hosted AI is suddenly back on the table, as far as I’m concerned. And thanks for the inspiration, Erik!