[Opinion] Chain of comprehension and vibe coding
It is inevitable that with enough time, one huge company will lose total control of the chain of comprehension over its IT systems resulting in catastrophic and irrecoverable loss of service. The practices we need to operate safely in this "vibe coding" space have not yet emerged.
That doesn't mean you shouldn't be using and experimenting with "vibe coding" (building without looking at the code) ... of course you should. Enthusiastically. However, experimentation must generate understanding, otherwise the system becomes unsafe. Experimentation without understanding eventually produces collapse.
It's like the early day's of Tesla's AC, amazing experimentation but not the "all safety abandoned" that some might imagine. Practical caution, rules of thumb and pursuit of understanding were still the norm. Without this, someone will fail.
To the gods of fate, when it happens, please don't let it be Amazon. I really like Amazon, the services are so incredibly useful. I would be so much happier if Meta, X, OpenAI or even Anthropic, vibe coded themselves into a early grave. I would be over the moon if it was Palantir.
From, Chapter 6 of Rewilding Software Engineering ...
"But does losing control of system comprehension actually matter that much? It's fine if you're talking about prototypes, but if you're going into production without that understanding, then you will increasingly be running the risk of becoming the next Knight Capital Group, a company that lost $440 million in 45 minutes because of behaviour within its systems that it did not understand. It's only a matter of time before history repeats itself. That's not to say that human understanding is perfect; it's not, as the Knight Capital example shows. It's actually an argument for why good engineering practices matter. By choosing a path of devaluing human judgement in the decision making process, then you are accepting the structural choices and values the LLM uses as long as functionality seems correct. In doing so, you are ignoring that functionality is only one subset of what goes into building a system. Comprehension is your last line of defence in a world where we have not yet developed the engineering practices to manage this safely."
Originally published on LinkedIn.