Vibe Coding - Or Maybe, What Happens When AI Hype And Reality Fail To Keep Pace
Dear Codeium Team,
I began writing this letter as a simple request to check into a charge that I believe maybe made an error.
But while I'm here, I thought I would share some thoughts about your product.
I'm copying this letter to a public format in order to articulate the views that I think are felt by many of your users who feel increasingly disappointed and sidelined by the neglect of what seems like basic QA in favor of a frenetic push for new "Waves."
Many of us are profoundly excited by your product and regard its potential, even, as transformative. But we are feeling increasingly disappointed and alienated by the way in which we, as ordinary users, are being related to.
FYI: I'm A Supporter
Firstly, and to explain why I am going to the trouble of doing this, let me provide my perspective on why I think that your product is more than just (another) entrant in the overcrowded world of AI SaaS.
I became interested in AI after discovering that the (now) humble ChatGPT could generate compliant-enough YAML to set up basic home automations.
Since then, it has been a whirlwind of exploration.
As both models and tooling become progressively more capable, the boundaries of what is possible continue to expand, and at a dizzying pace. This feels to me as if, for the first time in my encounter with technology to date, that we're seeing the first promising glimpses of what it can look like when technology works to make our lives easier and not more complicated.
I will admit to having spent more than one "all nighter" after the release of 3.7 simply marveling at what the ide was (kind of) capable of. While the results were inconsistent and sometimes hard to predict, the occasional first time successes kept the process worthwhile.
If the technology of yesterday was paying a handful of big tech companies to develop standardised tools, the vision of tomorrow which your tool suggestss is a much more democratic and interesting one in which everybody can develop the tools precisely fitted to their unique needs.
To Avoid Dot-Com V3, Product And Marketing Need To Be In Sync
But it's also for precisely this reason that I think that AI code generation, and the beginnings of this promising new era specifically, requires that vendors and users and enthusiasts work together and treat one another with respect. There is often less distance between us than it seems.
For this to happen, I believe that AI code generation needs to nurture a responsible articulation of its abilities which accords with an honest estimation of its current abilities and deficiencies.
I believe that AI code generation can be about much more than just developing MVPs or sometimes finished products. It can provide a novel and powerful interface for delivering examples-based education in programming.
But I don't see the value in overstating the maturity of what's on the market today. Doing so breeds dissatisfaction among users, who feel burned, and attracts justifiable charges that this is just another manifestation of AI hype.
When the gap between marketing promises and actual product capabilities grows too wide, users are left wondering if they're using the same product being advertised.
As a grizzled tech writer who chafes perhaps too quickly at "new and cool" language, your enthusiastic adoption of the term "vibe coding" to describe what can be achieved with your software arouses my natural antipathy.
So it's partilally for this reason that that I feel a sense of relief when I find that others are a little skeptical about the honesty and wisdom of describing the AI-human workflow that is fast becoming a reality as "vibe coding."
To my mind, calling it this not only greatly overstates the ability of this technology to generate high quality code unaided, but also understates the degree of expertise that is required to supervise these tools to achieve acceptable results.
The first generation of users is spending less time at present "vibe coding" and more time wondering whether Anthropic is still accessible and whether, while in the course of developing that wonderful new front end, the AI tool might have lost context and taken upon itself to write 10 unnecessary Bash scripts, a small encyclopedia of unnecessary documentation, destroyed some functional code, and then proudly announced that: "I have executed the completion tool having successfully completed the task!"
I have experienced moments of true "vibe coding" - Were a first time successes come to life perfectly. But this rests upon months of work in prompt engineering. And also a good degree of randomness. There are times when the acess to tool calling, the quality of the backing LLMs, or the state of Cascade seems to render even the most determined efforts worthless.
While I, as much as anyone else, would love to open up a beer and let my creative energy flow effortlessly into the IDE (okay, I'll admit it - I'm not sure that this is actually something that I want), I know from experience that in reality I would more likely need to interrupt my reverie in a few minutes to unstick Cascade from its latest bout of "stickiness." I would have something unbecoming of me like "WTF!? CAN'T YOU SEE I'M TRYING TO VIBE CODE?! JUST MAKE IT HAPPEN!"
AI code generation should be about democratizing technology creation, not just marketing the next shiny feature. When the gap between promises and reality grows too wide, we lose credibility. #AICodeGeneration #Codeium #DeveloperExperience
Growing Disconnect Between Marketing and User Experience
As is the norm in online communities my lack of real-world friends who have ever heard about a thing called "Windsurf IDE" or is aware that AI tools can do things like design functional websites is offset by passionate online communities on Reddit and Discord where those using the tool and excited about its growth exchange notes, workarounds and best practices.
During the last couple of weeks, I and many others have felt a growing sense of isolation and disgruntlement. Like many, I can't point to a single incident, but rather a series of down times and disappointments and let downs from the tool that were quickly followed up by another announcement of highly promising features.
Many are left scratching their head, wondering whether it wouldn't be more sensible to validate that the existing technology is working and remediate bugs before moving on to the next exciting set of features.
This, I feel almost certain in saying, has led to a growing sense of alienation among your users, many of whom are loyal and have expended significant resources in both using your tool and recommending it to others.
Challenges and Reliability Issues
It is clear to most who are engaging with your product that the challenge is inherent in attempting to pass huge amounts of tokens into the context required for this to work must be a monumental feat of AI engineering.
Most, including me, are therefore inclined to give you a wide berth in iterating towards success. I have explored the many competitors in the field (Aider, Cursor). My decision to persevere with Windsurf in spite of the ongoing frustrations has been less borne out of a lack of alternatives and more out of a strong belief that your model for context handling is the sensible one.
The UI of your product is excellent, and the insights you've shared into how it came to be are extremely interesting.
However, the description of the product as you have been articulating it seems increasingly at odds with the reality that many of us are encountering whenever we open it up in our computers.
Yesterday, for example, I noticed that the IDE is still unable to properly create a virtual environment in Python projects. Today I have a new bug in which code is being generated in the chat window despite clear instructions to the contrary. I have probably expended countless credits and tokens on circuitous fixes.
Despite the fact that the bugs daily users are encountering seem repetitive, fundamental, and highly disruptive, your marketing continues to announce feature releases and versions at a staggering pace.
As I take a long term view of the importance of AI assisted development, this makes me wonder whether the long term vision of the company and IDE does not see feature stability, reliability, and quality control as important values. If that is the case, it would be better to know these things now and move to alternatives.
Another point that I must raise to add myself to the list of those who have noticed this: your downtime reporting frequently seems at odds with what many of us are experiencing in daily use. Together, this creates an impression of disconnect and dishonesty that is deeply erosive towards the good feelings and enthusiasm that many, like me, first had when we began using your IDE.
A Call for Respect and Transparency
I believe it's widely understood that it will take some time before the necessary technology is in place for AI code generation to achieve a real degree of maturity.
Until that time arrives, I think it is vital to be honest with current and prospective users about capabilities.
You have developed a support base of users, including me to date, who feel enthusiastic about what can be achieved with the technology which you are working with. We are happy to be Guinea pigs, in a sense. But in terms that don't feel exploitative of our trust.
I believe that with some differences in how you relate to your users, this trust could be restored for the betterment of both your company and this important area generally.
With best wishes,
Daniel