Windsurf CEO: Betting on AI Agents, Pivoting in 48 Hours, and the Future of Coding

Choose Exciting Pivots

Pick a pivot that excites everyone on your team to ensure commitment.
Motivate your team with belief in your new direction to avoid immediate failure.

Transcript: Other Speaker So then what did you decide to do? So it’s like you decided, okay, we’re going to pivot. And when we work with founders, that’s kind of stage one. So you’re not half foot in, half foot out. So you had that conviction. We need to try something out. How do you figure out what was going to be the next step?

Varun Mohan I think we needed to pick something that actually everyone at the company was going to be excited about. I think if we had picked something that was what we thought could be valuable, but people were not excited about, we would ultimately, we would fail.

Pivot Criteria

Pick a new direction that excites everyone at the company for a successful pivot.
An opinionated stance, like recognizing GitHub Copilot’s potential, can drive innovation.

Transcript: Varun Mohan Think we needed to pick something that actually everyone at the company was going to be excited about. I think if we had picked something that was what we thought could be valuable, but people were not excited about, we would ultimately, we would fail. We’d fail immediately. We came with an opinionated stance of we were early adopters of a product called GitHub Copilot.

Irrational Optimism

Startups need irrational optimism to get started and uncompromising realism to adapt when facts change. These two beliefs may seem contradictory, but both are essential for success.

Transcript: Varun Mohan So this is where uh there’s there’s like an irrational optimism piece i’ve said this before but uh to the company but i think startups require like two distinct beliefs and they’re, They actually kind of like run counter to each other. You need this irrational optimism because if you don’t have the optimism, you just won’t do anything. You’re just a pessimist and a skeptic and those people don’t really accomplish anything in life. And you need uncompromising realism, which is that when the, when the facts change, you actually change your mind. Right.

Optimism and Realism Balance

Startups need both irrational optimism to act and uncompromising realism to pivot.
Balancing these opposing mindsets drives success in a fast-changing market.

Transcript: Varun Mohan Yeah co-pilot yeah so this is where uh there’s there’s like an irrational optimism piece i’ve said this before but uh to the company but i think startups require like two distinct beliefs And they’re, they actually kind of like run counter to each other. You need this irrational optimism because if you don’t have the optimism, you just won’t do anything. You’re just a pessimist and a skeptic and those people don’t really accomplish anything in life. And you need uncompromising realism, which is that when the, when the facts change, you actually change your mind. Right. And that’s a very hard thing to do both. Because the thing that makes you succeed through irrational optimism, like, you know, is exactly opposite to the things that allow you to be a very realistic company. So irrational

Surpassing GitHub Copilot Quickly

Windsurf’s initial model was worse than GitHub Copilot and free.
They rapidly improved by training their own model, eventually surpassing Copilot’s capabilities.

Transcript: Varun Mohan We might as well bet that we could do it. Were your early versions better than GitHub Copilot at the time? So our earliest version that we shipped out was materially worse than GitHub Copilot. The only difference was it was free. We built a VS Code extension. After Pivot, within, I think, two months, we had shipped the product and given it out to Hacker News, like posted something on Hacker News. And we built that out. It was missing a lot of key features. Like the model that we were running was like an open source model that was not nearly good as the model that GitHub Copilot was running. Very quickly, then our training infrastructure got better. So we actually went out and trained our own models based on the task. And then suddenly it actually got capabilities that even GitHub Copilot didn’t within two months. Basic capabilities. Now we’d find this hilarious that it was even state-of but our model could actually fill in the middle of code. So when you’re writing code, you’re not only just adding code at the end of your cursor, but you’re filling it in between two parts of a line, right? And that code is very incomplete and looks nothing like the training data of these original models, right? So we trained our models to make it actually very, very capable for that use case. And that actually allowed us to pull ahead in terms of the quality latency. We were able to control a lot of details

Betting on the Future

Varun Mohan says his company is always making bets on things that are not working. He is happy when only 50% of things are working because it indicates they are trying hard enough.

Making Bets

Varun Mohan believes that companies should always make bets on things that are not working.
He considers it a bad sign if 100% of the company’s efforts are successful.

Transcript: Varun Mohan And we have this free individual product. But I think one of the things about this industry that we all kind of know is the space moves really, really fast. And we basically are always making bets on things that are not working, right? Actually, most of the bets we make in the company don’t work. And I’m excited when I’m like happy when we’re when we’re, let’s say only 50% of the things we’re doing are actually working. Because I think when if 100% of the things we’re doing are working, I think like, it’s a very bad sign for us, because it’s probably like one of one of maybe three things. The first thing it is is like, hey, we’re not trying hard enough, right? That’s probably what it means. The second thing is we somehow have a lot of hubris, right? And the hubris is like, we believe everything we do is right, even despite the facts that are sort of on the ground. And then the sort of third key pieces here is we’re not actually testing our hypotheses in a way that like tells us where the future is going. We’re not actually at the frontier of what the capabilities and technology ultimately is. So we believed actually in the very beginning of last year that agents were going to be extremely huge.

Agentic Editors

Windsurf took an opinionated stance that agents were the future, unlike chat and autocomplete.
Varun Mohan believed that software would become easier to build, focusing on deep code understanding.

Transcript: Varun Mohan No, I think it’s a great question. So maybe the first point is at the time, actually, when we started working on Windsor, all the products were basically chat and this autocomplete capability. I think that’s basically what GitHub Copal was, what Cursor was at the time. I think we took a very opinionated stance that we thought agents were where the technology was actually going. We were the first agentic editor that was out there. And I think the biggest sort of takeaway was we didn’t believe in this kind of paradigm where everyone would be at mentioning everything, right? This almost reminded us of like the anti-pattern of what Google and the search engines were before like Google’s improved their product a lot, which was kind of like these landing pages That had like every distinct kind of like bucket of things you could search for. But Google came out with this very clean search box. Even Google at the time, you would need, you would get better answers if you wrote and or like site link. And now it’s gotten way better. Right. And I guess we had a belief that the software would get more and more easy to build. Right. And we would build from that starting point when we saw all the other players in the space making their product so configurable in a way that we thought was, I think, good for users now For where the technology was, but something that wouldn’t be unnecessary down the line. So we invested in capabilities like how do you deeply understand the code base to understand the intent of the developer? How do you how do you actually go out and and and make changes in a way that’s very quick to the code base? So we took the approach of, hey, instead of having this read only system where you tag everything, what happens if you could make changes very quickly? And that’s why at the time, we were kind of the first to do that. Now, if you were to ask, is that a very obvious decision now? I think it’s very obvious now. It looks very obvious. And this is where one of the things that I think is true for any startup is you have to keep proving yourself. Every single insight that we have is a depreciating insight it is a very very depreciating insight like technology the reason why companies win um at any given point is not like they Had a tech they had a tech insight like one year ago right actually if a company wins um other than the fact that they have you know monopoly uh you know it’s it’s uh it’s actually like a compounding Tech advantage that keeps sort of existing over and over again. And I think the example of this that I find most exciting is you look at a company like NVIDIA. If NVIDIA doesn’t innovate in

Depreciating Startup Insights

Every startup insight depreciates, so you must keep proving yourself. Continuous insights and execution are essential for survival and compounding tech advantage.

Transcript: Varun Mohan I think it’s very obvious now. It looks very obvious. And this is where one of the things that I think is true for any startup is you have to keep proving yourself. Every single insight that we have is a depreciating insight it is a very very depreciating insight like technology the reason why companies win um at any given point is not like they Had a tech they had a tech insight like one year ago right actually if a company wins um other than the fact that they have you know monopoly uh you know it’s it’s uh it’s actually like a compounding Tech advantage that keeps sort of existing over and over again. And I think the example of this that I find most exciting is you look at a company like NVIDIA. If NVIDIA doesn’t innovate in the next two years, AMD will be on their case, right? And NVIDIA will not be able to make 60%, 70% gross margins at that point, right? Even though it’s like one of the largest companies in existence right now. Basically having good insights to start with, you’re able to learn from the market and maybe compound that advantage with time. And that’s the only thing that could be persistent.

Other Speaker It sounds like a moat is, you know, we think of it as a noun, but it’s actually a verb.

Varun Mohan Yeah, something that could change with time, right? I also think for us, and I tell the company this, if we’re not continuing to have insights, and that’s why I’m completely okay with a lot of our insights being wrong. If we don’t continually have insights that we are executing on, we are just slowly dying. That’s like what’s actually happening.

Other Speaker I think the interesting thing is that it’s easier

Hybrid Retrieval Optimizes Context

Windsurf integrates keyword search, RAG, and AST re-parsing for best context retrieval.
Combining classical and AI tools improves precision in understanding large, complex codebases.

Transcript: Varun Mohan Everything is retrieval, augmented generation. But I think what people got maybe a little too opinionated about was the way RAG is implemented. It has to be a vector database that you go out and search. I think a vector database is a tool in the toolkit, right? If you were to think about what users ultimately want, they want great answers and they want great agents. That’s what they actually want. And how do you end up doing that? You need to make sure that what’s in the context is as relevant as possible. So what we ended up doing is having a series of systems that enable us to pack the context with the most relevant snippets of code. And the way we ultimately did that was it was a combination of keyword search, rag, abstract syntax reparsing. And then on top of that, using, as you mentioned, all the GPU infrastructure we have to take large chunks of the code base and in real time, re-rank it, right? As the query is coming in, right? And we found that that is like the best way for us to find the best context for the user. And kind of the motivation for this is because people have kind of weird questions. They might have a question for a large code base of, upgrade all versions of this API to this API. And if embedding search only finds five of them, it’s not of the 10, it’s not a very useful feature at that point. So we needed to make sure the precision recall was as high as possible, which meant that we used a series of technologies to actually get to the best

Run Tests to Evaluate AI Code

Code evaluation is done by running unit tests against generated code.
Breaking down testing into retrieval, intent, and execution accuracy guides improvement.

Transcript: Varun Mohan Interesting. How do the evals work? Yeah. So the evals for code are actually really cool. Basically, the idea is code, you can leverage a property of code, which it can be run, right? And we not only have real-time user data, we can put that aside for now, but we can take a lot of open source projects and find, I guess, commits in these open source projects with tests Attached to them. So you can imagine a lot of cool things we can do based on that. You can take the intent of a commit, delete all the code that is not the unit test, right? And then you can see, hey, are you able to retrieve the parts where the change needs to get made? Do you have a good high level intent to make those changes? And then after making the changes, does the test pass? You can do that task. You can mask the task. And by masking the task, it’s more like the Google task. And what I mean by the Google task is it’s trying to predict your intent, which is to say, let’s say you only put in a third of the change, but you don’t get the intent. Can you then fill out the rest to make the test pass? So there’s so many ways you can slice this. And each of them, you can break it down into so much granularity. You can be like, what is my retrieval accuracy? What is my intent accuracy? What is my passing? What is my test passing accuracy? You can do that. And then now you have a hill to climb. And I think that’s actually important before you add a lot of complexity for any of these AI apps. I think you need to like make a rigorous hill that you can actually climb, right? Otherwise you’re just shooting in the dark, right? Why would we add the ASG parsing if it’s unnecessary? Actually, it’s awesome if it was unnecessary. Um, right. I don’t want to add a lot of complex stuff to our code. In fact, I want the simplest code that ends up having the most

Evals vs Vibes

Balance rigorous evals with intuition, especially for complex AI apps; create a measurable "hill to climb" before adding complexity.
Vibes are valuable for easier things but harder to eval, and user data helps to build evals after the initial vibe.

Transcript: Varun Mohan What is my test passing accuracy? You can do that. And then now you have a hill to climb. And I think that’s actually important before you add a lot of complexity for any of these AI apps. I think you need to like make a rigorous hill that you can actually climb, right? Otherwise you’re just shooting in the dark, right? Why would we add the ASG parsing if it’s unnecessary? Actually, it’s awesome if it was unnecessary. Um, right. I don’t want to add a lot of complex stuff to our code. In fact, I want the simplest code that ends up having the most impact. So the evals were actually really, really critical for us to make a lot of these investments at the company.

Other Speaker How much of the development that you do is basically driven by improving the scores on the evals versus basically vibes-based? You guys are all using Windsurf yourself. You’re getting feedback from users all the time. And then you have just like a sense that this thing is going to work better. And then the evals are just sort of like a check that you didn’t screw up something else.

Varun Mohan It’s a little bit of both, but obviously for some kinds of systems, I think evals are more important than vibes, but like more easy than vibes. Just because for the system that basically takes a large chunk of the code, chunks it up and passes it to hundreds of GPUs in parallel, giving you a result in one second. It’s very hard to have an intuition of like, is this way better? Because that’s a very complex sort of retrieval question. But on the other hand, there are much easier things that from a Vibe perspective are valuable. What if we look at the open files in a code base? This is actually a harder thing to eval. Because when you’re evaling, you don’t know what the user is doing in real time. This is one of those cases where having a product in market helps us a lot, right? And we’re able to take a lot of user data on how people use the product to actually actively make the product much better. So that’s maybe starts with Vibes. And then after that, you can build eval, right? So it’s a little bit of both, basically.

Other Speaker I think a lot of chatter on the internet

Manage AI Changes Carefully

Provide clear intent and commit frequently to manage AI-made code changes.
Revert mistakes quickly to maintain trust and productivity using Windsurf.

Transcript: Other Speaker Happen. But how do you get those precise changes? What do you do? How do you feed the system? How do you become a…

Varun Mohan Shout at it with all caps, right? Yeah. No, so I think this is one of those things where I think you kind of need to have a little bit of faith in the system and let it kind of mess up a little bit, which is kind of scary because I think A lot of people, for the most part, they will write off these tools really quickly. Obviously, no one at our company would write off the tool because they’re building the tools themselves. I think people’s expectations are very high. And maybe that’s like the main piece of feedback I’d give, which is that, you know, our product actually for these larger and larger changes, it might make 90% of changes correctly, But if 10% is wrong, people will just write off the entire tool. And I think at that point, actually, probably the right thing to do is either revert the change, we have an ability to actually revert the change, or just keep going and see where see where It ultimately can go. And maybe the most important aspect is commit your code like as frequently as possible. I think that maybe that’s like the big, big sort of tip there, which is that, you know, you don’t want to get in a situation where you’ve made 20 changes and on top of that made some changes Yourselves and you can’t like revert it. And then you get like very frustrated at the end

AI Will Multiply Developer Leverage

AI has rapidly improved in complex tasks like math olympiads, from poor to nearly expert.
AI will soon multiply leverage across all software development phases, drastically speeding work.

Transcript: Other Speaker How is it going to evolve?

Varun Mohan There’s probably a lot of people that think the vibe coding is kind of a kind of a fad but i think that’s going to get more and more capable with time i think you know whenever i hear someone Saying hey this is not going to work for this complex use case it’s it’s like it feels like a luddite saying something it’s like if you look at the way these these uh these ais have gone better Sort of year over year it’s it’s actually astonishing it’s astonishing like i’ll give you an example of of something that i held like kind of near and dear to my heart which is you know This math olympiad called the Amy. I used to do that in high school. I was very excited about how well I would do it. My high score was somewhere close to 14.

Other Speaker That’s a very high score.

Varun Mohan But the crazy thing is that was one of those things that I thought, oh, wow, like the AI systems, they’re not going to get anywhere near as good. And beginning of the year last year, it was probably like well under five, maybe. And now they’re, you know, the average that OpenAI has put out is like it’s getting 14 and a half to 15, right, for 04 mini. So it’s almost like you have to keep projecting this out, right? It’s going to get crazy. And basically every part of the software development lifecycle, whether it be writing code, reviewing code, testing code, debugging code, designing code, AI is going to be adding 10 times the amount of leverage very shortly. It’s going to happen like much, much more quickly than people imagine. Going back

Use Open-Ended Interviews

Interview candidates with open-ended system design and algorithm problems.
Use these to assess problem-solving and curiosity beyond code generation capabilities.

Transcript: Varun Mohan Like kind of hate these tools or not. There are still some developers that do. And obviously, if you do, we’re probably the wrong company to kind of work at. But also at the same time, we do have sort of interviews in person where kind of on site where we don’t give them the AI and we want to see them think, right? It would be a bad thing if ultimately when someone needs to write a nested for loop, they need to go to chat GPT. Right. And I’m not like that’s fundamentally because because it just it just feels like that is a good proxy for problem solving skills. And I think problem solving skills are just at a high level still should go at a premium. That is that is the valuable skill that humans have.

Other Speaker A challenge that a lot of companies we’ve talked to have had that we’ve even had ourselves is that Windsurf has gotten so good that if you give people windsurf it’s difficult to even come Up with an interview question that windsurf can’t just one shot where you know anyone can do it because you literally just copy and paste the question into windsurf and hit enter and

Varun Mohan So you’re not really evaluating anything yes so i actually think that’s true and it’s it’s you’re totally right There’s very few problems now that something like an 04 mini is not able To solve now. Right. I mean, if you look at competitive programming, it’s just, it’s just in a league of its own already at this point. The crazy thing is interviews by nature are going to be kind of isolated problems, right? They’re by nature, because if the problem actually required so much understanding to do, you wouldn’t be able to explain the problem. So that’s like perfect for the LLMs where you give them an isolated problem where you can test and run code extremely quickly. So yeah, you’re totally right. Like I think if you tell, if you only have algorithmic interviews and you let people use the AI, I don’t know, you’re not really testing anything at that point.

Other Speaker Does that mean that you’ve gone away from just algorithmic questions and you ask different, like much harder questions that are actually well suited to being able to use an AI?

Varun Mohan Yeah, we have questions, obviously, where that are that are both system designing plus algorithms, algorithms related. But these are questions that are fairly open ended, right? There may not be a correct answer. There are there are trade offs that you can ultimately make. And I think what we want to do is just see how people think, right, given different trade offs, and different constraints, right? And we’re trying to validate for like intellectual curiosity, right? And if someone ultimately says, I don’t know why, why, why, that’s totally fine. As long as like they’ve gone to a depth that we feel kind of shows, you know, kind of interest, interest and like good problem solving skills, if that

Keep Raising the Performance Bar

The AI coding product must continuously improve beyond foundation models.
Even small improvements over strong baselines create significant competitive advantage.

Transcript: Varun Mohan This is like, yeah, the company, as I mentioned before, it’s a moving goalpost, which is to say today, if we’re generating sort of 80, 90% of all committed software, I think when the new Model comes out, we’re going to need to up our game. We can’t just be at the same same stage. Maybe we need to be generating 95 percent of all committed code. And I think our opportunity is the gap between where the foundation model is and what the what what 100 percent is. Right. And as long as we can continue to deliver an experience where there is a gap between the two, which I think there is, as long as there’s any human in the loop at all in the experience, there’s A gap, we’ll be able to go out and build things. But that is a constantly transforming sort of goalpost for us, right? So you can imagine when a new model comes out, maybe the baseline on what the foundation model by itself provides has doubled. The alpha we provide on top of what the base model provides needs to double as well. It feels very, like for me, the reason why this is not the most concerning is let’s suppose that you were to take the foundation model and it’s providing 90%. It’s reducing the time it takes by 90%. That actually means if we can deliver one or 2% more percentage points, that’s a 20% gain on top of what the new baseline is, right? Like I guess 90, if 90 becomes 92 or 93, which is still very, very valuable right at that point, because effectively the 90 becomes a new baseline for everyone. So I think basically the way we sort of operate is how can we provide as much additional value as possible? And as long as we have our eye on that, I think we’re going to do fine.

Focus on Specific Workloads

Focus on doing one thing really well within software development.
Find specific workloads, like Java migrations, to build a successful AI coding startup.

Transcript: Varun Mohan Seen a lot of things that I think could be particularly interesting. I don’t think any of these technologies we’ve really adopted, but there’s so many different pieces of how people build software. And I’m not going to say niche, but there are so many different types of workloads out there. I’ve not really seen a lot of startups in the space that are just like, we do this one thing really, really well. I’ll give you an example. We do these kind of Java migrations

~/adi