Reshaping Investing With AI

Thank you. That’s very rare at Microsoft. Actually, I was working for Ken and Scott. Usually, CTOs at Microsoft are product-specific.

But in this case, I was partnering with CEOs that report into Satya to really think about how they think about their product portfolio and what they can do for that. It was a phenomenal time and it’s extremely interesting, investing and opening eye and having that relationship, and then applying that across the enormously of the portfolio at Microsoft. Perfect. We will come to the second part of your story in terms of venture capital.

We will come in the second part of our discussion, but let’s stay on the Microsoft discussion. You very early on thought about how to bring technology into products. You did this, as you described, across Microsoft’s portfolio. What has changed?

We see now the ability, Opus 4.5 launched beginning of the year. How does product management change? How does the software development cycle change? AI is completely upending that piece.

I think it’s really funny because as technologists, of course, we’re going to apply it, but we’re going to have to disrupt ourselves. We just can’t help it. But if you talk to, look, there’s a lot of disagreement of how it works in practice because the diffusion piece, so taking technology and actually applying it in large corporations that are not necessarily heavily tech first is quite difficult. But if you look at technology first firms, the startups, the AI labs themselves, those really start to lean on and a lot of them, will claim that they’re writing 80, 90 percent of their code via AI.

There’s a replacement event that is beginning to occur, which is happening really rapidly. We’re swapping the horse for the car, and all of a sudden, all of us who used to shoe horses, which are the software developers, we are going to have to adapt to this new world and find a way in it. Fascinating. May I jump in here?

This is like the horse and the car. If you use an analogy, we don’t need horses anymore. Do you think in the future we will need software developers? Well, I compared software developers more to blacksmiths, I think.

Okay. I think we’re probably not going to be just going to a car, going to spaceship at this point. Yes. But I think software developers are going to take a very different role, a much more abstract role just like when we went from punch cards to compiler, to assemblers, to higher order languages.

Yes. We’re upping the level of obstruction. Yes. So that we just do still needs in order actually, to have a decent, vibe-coated program, you have to be quite specific about how you want it to be stitched together, how you want it to be architected.

You have to define it really, really well. So I think what’s happening is the technologists are going to have to turn into technical PMs. That role is going to, at least for a period of time, until, you know, everything is just, you know, we’re in singularity mode. Until then, that is going to be the craps of the job, right?

And I think, like, to bring this together, I think, yes, we see models that can code. That means coding as an activity is probably going away. What won’t go away is software. Software is still needed.

So somebody needs to construct the software. I just told you in the green room about the story which I had, right, that I wanted to say, I wanted to sell my company, and I created on top of the search company I had built, I created a geo-generative engine optimization company. It took me two days to code it and two months to actually get it then with manual work stable. We do need the software.

It’s only way faster for us to iterate around it. No, that’s absolutely right. In fact, it’s going to massively expand, right? If we started thinking about, what’s going to happen to the software, oh my God, it’s going to be exponentially more of it, right?

And then, by the way, it also needs to be high quality, right? Which is another huge problem, right? And then as you start thinking about managing, maintaining it, right, that is a whole other domain and integrating that. So there need to be, there still will need to be people that are going to be around it shepherding and helping, understanding, just like you need people around now, you know, driving cars and all of the industry that happens around that, right?

So that event is absolutely occurring. If we think about the software stock, there is probably, there is this need to structure it, the architectural need. There is then the need to generate it, and we can talk about slop in engineering, but then to maintain it, less slop, more quality code is easier to maintain. But again, this could be replaced, and then to actually have oversight about it.

AI is always about quality and control, meaning having the right quality on one side of code, as well as having control that the code is actually doing what you want it to do. Yeah, that’s verifiability, right? And you see, you see this in terms of, actually, how models are developing now but sadly not everything is verifiable right software not fully verifiable only partially so you know we that problem is not yet solved you know by ai itself what did you think about open claw maybe quickly explain what open claw is mold book open claw big hype at them like used to be big hype we just saw open ai buying the buying it out essentially yeah buying the founder the creator right yeah yeah i mean so so look it’s a it’s essentially you know uh open libraries for creating agents that really easily creating agents on your own so that everybody sort of uh people who who don’t code can can do that obviously can create a lot of damage so you probably have seen a bunch of people you know saying that oh my god i’ve lost all my photos or things like that you know some scary lessons there to start um and then of course you know you probably have seen the network of open claw agents you know talking to each other and how that’s evolving it’s the the phenomenon i think really is important because it opened the doors to really people starting to feel like it’s i think that the gap between those of us who are in the industry and everybody else is pretty wide and widening and events like these are they they start to close the gap for a little bit um partially because they now enable everybody else to play with things like that partially because they bring it into the design guys so it’s you know there isn’t necessarily anything particularly you know novel there except for how you know how it’s packaged and distributed and who can who can access it easily this despite the fact that you know you could do it without something like open claw i very much yeah i’m very much like the point on packaging and distribution because that we will get a tool set to create agentic workflows there was very clear since two years at least right and um you saw a lot of companies which that created tool sets for a very specific purpose you had suddenly companies that said i do everything around um quickbooks on don’t not quickbooks but around financial flows and you had companies who said i do everything around audit so suddenly everybody was building their workflows and then you had general purpose platforms so it was clear that this will come what was i think impressive is that it was so easy to set it up which is um um like setting up the quality like good quality stuff was easy what was i think impressive was that it was so easy to set it up which is um um the thing that was still not there is control again ai is quality and control meaning yes it can delete everything and you have no control over it you know you’re completely right uh it’s it’s the uh ability you know user interface you know is still the thing that changes everything right the the reason here it’s not like we don’t have a lot of frameworks you know and all the big companies have a bunch of tools to build agents you know all of the ones you know you can name from google to microsoft to you know anthropic to open the eye they all have stuff but here something that has a really easy way and comprehend easy to comprehend way for uh you know for a person who is not necessarily in the know to play with it person who is not necessarily technical to play with and how important that is right and then on top of that creating hype and uh gamifying it so that it’s interesting and people want to play with it right so those two things it’s brilliant i think the as as lms came out very soon ux designer said this is awesome but the interface is the biggest problem and we see that story holds true till today we can design workflows but it becomes very complicated to sign up i recently tried to sign up for the sandbox of sierra and only to learn it doesn’t work i cannot easily sign up others offer sandboxes but then it’s still too complex meaning we have not yet cracked the approach how do we steer beyond language because language is the worst user interaction we can imagine oh i mean we seem to do okay as humanity but it’s very low fidelity it’s low fidelity exactly quality and control what did you exactly mean with what you said like um exactly it’s ambiguous right which is that’s the reason why we humans tend to sit in long meetings forever that’s right but but that’s again where uh programming language is a really important tool on this journey right because if you can reduce human language a big ambiguous human language to the machine to effectively program it to the machine it’s not going to work so we have to have a more advanced way of programming language which is a lower level abstraction and if you can have the model do that and now you can solve a lot of problems which is exactly where it’s going to go once the you know the programming problem is solved it’s now becomes so much easier to solve other problems that are you know have much more ambiguity now if you let’s think back four weeks or like six weeks like we get into the new year we have those new launches out there we have the new launches out there there from Oppos. What was your thought process there?

What do you as VC in this case think when you see that suddenly it becomes so easy or it’s a new level of coding excellence? We sort of saw it coming, right? So reinforcement learning, so we started with pre-training, right? Pre-training emerged, it morphed slowly in the reasoning models, what you would call.

And then we already saw because we see things a little bit earlier because we’re tracking on papers, we’re going to new reps, we’re sort of in the know. We have some incredible people on staff who are researchers themselves. So we saw the RL wave coming, right? So we saw this transition from ROHF to ROVR.

We saw actually labs emerging in that space specifically because researchers saw that this is going to be a big deal. So there’s a little bit of expectation. What we didn’t know is how much it’s going to up the game for current players, right? Because we knew current players were running ROHF gyms already and working on that.

So when the models came out and they sort of showed up, the quality was just, they just became so much more useful and not just for coding, right? They became so much useful for just general tasks, like I wrangle spreadsheets going down, you know, like I’ve never done before, you know? So that part was a surprise of how quickly they’re moving. And from what I understand and, you know, from what we see, it’s just the beginning, right?

So we’re still kind of early into this ROH wave. So we can expect the scaling to kind of, long to continue and capabilities to increase. Yes. Now when you, what’s your kind of prediction?

And you had a framework, have a framework in mind. You talk about accuracy, memory, open-endedness. Talk me a little bit through, but before you do this, I just want to do a quick public announcement. No worries.

It’s not an advertisement here, but for, I see questions coming in and I wanted to tell everybody, please enter questions as you follow our discussions here. I will see your questions. I bring them in if they are appropriate. Think about the future of all the models which we are discussing, Leela and I.

Then think about how AI is used in a venture capital context. And then at the end, we will talk about AI and product. So if you have questions, ask them, I will bring them in. Now Leela, back to you.

Okay. what they’re going to be able to do, even though I think it was very apparent how incredible they already were at that point, right? And how that generalization was already happening. But, you know, even as a, you know, I am a very big positive person, and with respect to technology moving forward and knocking out barriers, but even I had all these questions about, are we going to be able to solve this and this, right?

And back then it was, you know, are we going to be able to keep them on rails? Are we going to be able to have them give accurate answers? Are they going to be able to do mathematics, right? Like, all of these early questions about them, I think accuracy was huge, right?

And when the guys came back with, you know, them beating all of the, solving all of the tests, right, that was sort of the early sign of what’s to come. And that is, we’re going to set this, we’re going to keep complaining, and we’re going to keep solving those problems, you know, all in a row, one at a time. So our row right now is solving a lot of accuracy problems, because it’s based on verifiable rewards, right? VR stands for verifiable reward, right?

So you can say, you know, this is the answer to the problem. Now go and figure out what the answer is. And then you can go and figure out what the answer is. And then you can figure out how to get to that solution, right?

It’s sort of, think of this as the, think of AlphaGo or AlphaZero actually, as the sort of the grandparent of that. We have a true north. Whenever we have a true north, then we can train a model to figure out true north that’s very traditional machine learning approaches. Exactly.

Exactly. So what we need is just an answer, and then we can figure out how to get to the answer. Yeah. Right?

That’s, so that’s the thing. So I think that’s a really good point. I think that’s a really good point. So consider that solved.

I think that’s going to yield a lot of accurate results for questions that we know answers to. So what’s next? What is still unsolved, right? What should we be thinking about?

So memory is still unsolved, right? The working memory, what I would call. And we have like large context windows. They have problems.

We have sort of databases on the side. Those have problems, right? The model does not have, we have long-term memory and then we have short-term memory. That piece is not solved today.

And there’s, there’s like, you know, there’s a lot of different sort of, you know, there’s labs and different sort of trains of thoughts of how to solve this, you know, and I think we’re going to knock it out. That’s, you know, it’s just a matter of time. Right. And then after that, how do we solve problems that we don’t know the answers to?

We should, that is the most interesting thing. This is where, in my opinion, we’re going to talk about AGI. That’s AGI, right? That is where, you know, we don’t know the answer.

We know the problem that we want to solve and we need to find the answer to it. And you see this in biology and chemistry. I mean, you see it in even, you know, any, any discipline actually will have that, right? Mathematics for sure, right?

And very, very concrete things. And this is where, you know, things like open-endedness comes along. I like the framing. The framing essentially is we have accuracy on one part.

And as long as we know what is true, we can solve accuracy. We have an accuracy as well obviously. And I say this from search and discovery. If you’ll go to a retailer and look for a black bag, there are 15,000 black bags that would fit officially.

So the accuracy problem is a little bit fuzzy here, but we have a feedback loop. So accuracy was one. Second is memory. And third is open-endedness.

Let me ask to the open-endedness one. And I, this is, it’s a, it’s an interesting discussion, which I used to, you know, I used to talk about a lot of this, but I’m going to talk about this a little bit more. And I used to work for Google Health. I helped to set this up.

And I was at JP Morgan conference this year. And when you ask about how AI is used in healthcare, you will realize that most people do not yet use AI in open-endedness. They use AI for workflows. It’s kind of like test it out.

I have my test plan. I have an agent which reaches out to a patient. I sign off the documents. I find people search and discovery.

I supercharge the process, but I don’t do open-endedness. What meaning going to JP Morgan, I was not very impressed what I saw. But if you, you from, as a VC have a completely different view here, what do you see there? Well, I think, you know, open-endedness isn’t about, it makes sense that at JP Morgan conference, you heard this, right?

Because there’s a big divide between where the world is as AI is getting, the adoption of AI is, right? And there we’re quite behind in many ways because we still don’t really know. We don’t have clear patterns of how to deploy AI in a very safe way, right? Is this, you know, this is still fuzzy, right?

The blueprints and the patterns have not yet been set and the, you know, very robustly, I would say, right? So, so this is a very high bodge kind of situation. And I think in the next year, this year, I think we’re going to start seeing, you know, really adoption start taking hold and mass. And this is why we saw this, you know, 95% of the projects fail, et cetera, People are not ready, et cetera, all of that.

Right. So that’s open-endedness is way out there on the horizon. You know, it’s, we are in the Arrell wave right now of model development. So open-endedness just started.

It’s, I think the, the deep mind paper came out last year. I want to say that really sort of made waves and, and it’s unexplored still. So only the most sophisticated labs are dealing with that right now in terms of how they are playing with their models and how they’re setting up, you know, their, their test fronts. It’s especially important for life sciences.

And you’re completely right talking about JP Morgan because most life sciences problems. Don’t have a solution. They don’t have a solution or we don’t know the solution, but they have really big problems, right. That they want to sell.

So, so designing proteins to target, for example, is, is an open-ended problem, right? We know what sort of we want to solve, but we don’t exactly know, you know, exactly, exactly how, what, needs to happen because the complexity is too high. We talked a lot about the excitement about new models. You gave us a framework to think about where models are going.

This is cool. Now I am, there is a paper from professor from Wharton. Her name is Hamsa Bastani, and she kind of looked at the different models and used a metric to describe the efficiency of the model. And there are different matrices, obviously, as you normalize this, and then she kind of tried to fit it to a curve.

And, and she said that curve is actually more an S-curve. It’s, it’s flattening off. New models are all better, but it’s incremental. That’s still.

It means it keeps, it starts going up again. Well, at some point. That makes sense, actually, this tracks, right? So if you think about pre-training and people talking about pre-training hitting a wall, et cetera, et cetera, right.

Which sort of, you know, the model builders will tell you, no, there’s no wall, et cetera, but, you know, you think about the data, right. You know, and, and there’s only so much data on the internet. We exhausted that, right. We get user generated data.

We use that. Then we get synthetic data. We keep using that. But until IRL started to hit, I think IRL is the second leg of that curve, right?

That upper curve. So we’re starting to get on the back onto it. So maybe if, this is, this is accurate, you know, I haven’t read that research, I’m sorry. This is accurate.

That was sort of tracked. Yeah. There’s also, there’s also a heightened level of machine learning in general in general in general in general in general of what has happened has been inspired in the last sort of year year and a half meaning for you there is the rl um is essentially um going into recursive development this is where we will need to we’ll see the next boost of models we already have so this current boost is already coming from got it now how do you and like there is um daniel uh from kansas uh he asks um as an investor do you actively push portfolio companies to invest in ai because like obviously you guys have way more and if we are seeing now the trend is this something um you you actively train or push your portfolio companies towards so first of all i’m not as presumptive as i am but i’m not as presumptive you know ultimately the um the founder is in charge right um and i’m there to my job is to be a sparring partner and help think through especially strategic issues so we have almost you know we have over 500 companies i think that are active in the portfolio and uh you know you don’t get to interact with every founder of course but for especially for where it feels strategic um you absolutely want to have the conversation brainstorm with the founder of how to apply how to accelerate them there’s no question about that and you actually see the those that act fast and those that act slow i love to take the example of databricks which is our portfolio company and snowflake which is another great company very very similar sort of um capabilities but databricks had a major breakthrough when they embraced the very very early you probably remember mosaic purchase uh helped help them accelerate and really pivot into the wave and that just made such a massive impact to the company and and evaluate you know obviously valuations etc which is fascinating right because databricks came from a traditional data company and then it moved to ml and now it moved to generative ai which is talking about your microsoft experience it’s it’s de facto an amazing story about how to adjust to new technologies it’s actually extremely difficult right so you kind of have to constantly be in this you know there’s a saying uh which i don’t love but it’s very very true only the paranoid survive and it couldn’t be more truer than now because technology is moving so fast that you can start a company six months ago and be disrupted already so you have to constantly be learning and constantly thinking about what is coming next and how do you need to get ahead of that but if you achieve some level of takeoff you have to achieve distribution that you’re out there in front of users in front of customers you’re getting invaluable data points that help you get better than anybody else it’s still true just like it was in the past that that signal is gold right that signal is what’s going to help you differentiate and the faster you move bigger you get the more you tear tear away from the competition the more you become indispensable got it awesome yeah let’s move let’s move over to the vc side of things um i am machine learning is meant to find tiny clues in any kind of data set meaning if we have an image of a cat a machine understands the tiny clues what makes a cat a cat and if we turn this around we have generative ai so far so easy now if you’re looking at um a vc a vc tries to find tiny clues about is this person an investable person is this an investable idea do i have here the right network to help grow the company um how is ai used to support vcs um yeah so it’s said that by now over 80 percent of private equity and vc firms already use ai so this is definitely has become a fundamental component for most firms at nea we’re using ai heavily in the process you know everything from you know the beginning sort of sourcing uh and and uh finding sort of that needle in the haystack helping uh investors do that all the way through due diligence and support um and we started that we we always had technology obviously underpinning that we have a ton of data because we’ve been around for gosh nearly half you know half a century um and then so that really differentiates what we what we’re able to do but most importantly you know ai came along and enabled us to do all these things where we have a bunch of agents working and looking at different signals from different sources and then we’re able to do all these things and then places right anything from github to linkedin to um you know just blogs and conversations online you know you name it and of course you know the the standard uh you know harmonics and crunch bases right so that accelerates our ability to respond and react and get in front of the founders early and then uh you know of course throughout the process it just accelerates human judgment uh it helps us really you know make better decisions make the faster understand better um so that’s absolutely at this point is uh on you you wouldn’t be able to do the job without that i would say does this mean and um there are two questions from the audience to this um one is from helen that actually asks on um uh the feedback cycle if we are um only looking at heat scores and celebrity founders whether this becomes a reinforcing cycle and the other question is how do we know that we’re not going to be able to do anonymous that asks whether um actually we should not reach out to vcs anymore we just need to wait that a vc picks us up because they see us and if they don’t see us if we reach out they wouldn’t talk to us so there’s definitely a hack algorithm hacking that’s going on so for example if uh if you you probably notice that if you know a vc friended you on linkedin you all of a sudden get more vcus you know looking at you that’s because the algorithms are picking that up and by the way that can be misleading they also i just i don’t want you to uh i don’t want people to over to use uh algorithms uh with abandon and you still need judgment uh because for multiple reasons first of all ai is a regression to the mean you have to keep that in play uh you have to keep that in check ai is biased right ai is looking at previous patterns right so you have to constantly challenge yourself with this uh to answer your question more directly of course you can you still reach out of course the relationships still matter and they’re going to continue to matter because at the end of the day you’re making a relationship you’re making you’re making decision based on trust of who you trust both as partnership and as an individual partner that you’re going to be working with or partners that you’re going to be working with and you know in any case you’re going to be working with a whole team you know and that is a human connection right so don’t completely neglect that i was actually fascinated so i sold my company in september and i announced that we we sold and so on and so forth and what happened afterwards was actually fascinating a lot of vc companies reached out to me um very i would say heartless automated calls these were humans these weren’t any agents but they clearly followed scripts trying to figure out what are you doing now can we meet for a coffee is something what’s your next plan i i thought that was very funny well and i think it’s also really disheartening because when you get these emails you kind of you don’t want that and i think again going back uh let’s do that user experience right i am not going to answer a lot of those but and this is why for us we always have a human in the loop right if you if we don’t have ai sending blank emails right the partner needs to look at the deal look at the founder and want to talk to them right and and that’s by design because it is still a human business you’re going to build a company together you know and that is so important who you’re building the company with why would you want to you know not know that person right yeah totally makes sense um i think the uh the choice of a partner is something which um early on founders very often disregard and um and it’s like a marriage um even with a vc a hundred percent so i think that uh for for us especially the technical founders um it’s really important for them uh who is on the other side of the table have they have they done do they understand my space do they understand my problems do they do they know where to go and this is where that competency is really really matters lila when i come to you let’s say i pitch to you um lutz corp uh i i’m not sure if you’re going to tell you about ai it’s amazing you’re ready i pitched to you something what would you look at like no no we have had dinner together we we know each other so this is not the point like take not let’s corp let take um whatever corp funny german uh accent guy corp right so what would you look at if you don’t know me you know i think obviously look fundamentally for a few things if it’s really early they look at the team they really look at can these people you know do they have what it takes to get there right do they do they have the stamina do they have the desire do they have the big vision can they have they work together i can do they trust each other all of that that team dynamic background you know a lot of a lot of vcs will tell you actually talking about finding german accent is you know they look for immigrants for example because you have to overcome adversity because you have to learn to exist and and thrive in a new environment you know that shows resilience so we look for that you know so that’s those are the early days and then i think as the company progresses there’s still metrics you know we start to look at metrics the metrics however have moved dramatically in this way right so i think the closest we can look at is are the sas metrics of the past right so there’s still people are still looking at them by the way i actually personally question if they’re looking at the sas metrics of the past because they’re exactly right because for example ar is that still going to exist in the play in that at the time when we move from let’s say subscription to outcome-based pricing right but for now people look at that right uh we look at uh you know burn rates and you know gm and you know durability of revenue and things like that right so once you can start looking at the numbers you look at the numbers of course got it but for sourcing you um do you have any signals that help with sourcing in terms of so you don’t know me you don’t know um a funny german speaking guy corp um the guy um you see me the first time are you analyzing my linkedin network or a hundred percent you bet yeah i’m analyzing everything i can get on you everything i can find you know either through private networks or through public channels you know i’m looking at what you’ve done in the past your education you know your sort of life journey previous companies that you’ve done you know did they exit how how did it work out you know your team of course you know how long you’ve known each other yes that’s a hundred percent and then there are companies that go way deeper than that you know that uh that even just kind of those things right and then if you’ve started if you’re developing in the open right if you have a project going absolutely that data will come up as well right so how many r do you have on github how many adopters do you have of your solution so all of that is now available and what happens is you know that all gets dumped into you know one big databricks you know uh database and then you know ai uh does its magic on top of that so to say i want i won’t disclose too much no it makes sense and you are and how important are those metrics for you if i’m thinking now like let’s compare to a pilot a pilot is very strict on all the different metrics they are using before they take off or is it more nice to have i think it’s a must-have nowadays because you get the signal earlier so you’re more competitive right so these outliers that you’re watching by the way our investors actually put in what they are specifically looking for so the algorithm and the investor work together and sort of it’s it shapes their agent you know to uh to basically be more precise so you know let’s if you’re investing your agent will take a direction from you you can give it feedback so it’s fascinating so the investor trains the individual algorithm for searching that’s right that’s right so you can say like you know what i don’t like that lila woman don’t bring me more records yeah understood so you can you can absolutely train your own algorithm you can you can give it feedback you can correct it i remember you know vinod i used to actually it was his idea uh like 10 years ago i think when alexa first came out he used to lament that he can’t tell alexa that whatever she said is stupid you know and change that it’s funny that you still can’t do this and we kind of decided that that’s a fundamental piece of providing feedback to to our agents so so they will they will look it up they will give you an indicator that something is happening earlier than you normally would find it so a lot of times we’ll pick up the signal you know as soon as the founders left just leaving you know their current company this is what you’re saying right because your company has solved that’s a very strong signal that’s why the algorithms have picked it up but there are weaker signals that you can identify and as soon as we identify those signals you know the the you know the flag will go up and the investor can look at that particular opportunity and basically say this is interesting i actually want to double click on that one so that’s that’s really really valuable and then you know so so the noise is enormous there’s no chance my little you know internal neural network can deal with that and my external neural network is actually really helpful now yeah thomas actually asked about how you work with those cycles and this is because you just talked about the signals what does it actually mean if you get um somebody who tells you i can now very early on right i can do summarization with a model or i can answer with a model and then you suddenly have those companies and all of them got flatlined once open ai came out so open ai there is a saying that many companies will be dying because open a make this part of their system how do you how do you deal with this uh look the this that’s sort of the self-disruption cycle is is re is always happening it’s just happening sooner now than ever before so you have to pay more attention to how quickly and what what threats what risks happen right where ultimately risk managers more than anything else will look at what can what can happen in the ecosystem that will make it more difficult for this company to realize that it’s its vision right and what things can help it right because we want to help and and we have tools to help that company right so when we were investing even in space as as soon as you know sort of the foundation models appeared we were already looking at what is a called blast radius right this this guy these things are very powerful right and you don’t want to be in the blast radius of those what is defensible and what is not so you always think about this defensibility you won’t always get it right but you will you know if you if you think that way you’ll definitely get it right more often you know so yeah it’s fascinating i went through the same story by myself right i kind of created fine-tuned models to do search and discovery for retail that means some like every website has a search and i looked at the traditional companies the algolias of this world and i said like i can do this better i have a better technology and that it means in itself okay i can i have higher conversion rates i make more money this is all nice but how long will it take till the traditional companies like algolia catch up my estimate was it will take two to three years now the cycle is way faster right it took they are still not there i’ll go you are still limping behind the but they have caught up big time one of the reasons for me then to decide okay i better sell my company yeah no i mean the cycle here’s an example for you not an example but statistic for you the companies today and some of this you can say is a little bit messaging, but the companies today are eight times more likely to get to 10 million AR within the first year than in any of the previous cycles. Eight times, that’s insane.

Eight times. Wow. Yeah, that tells you about the speed of disruption. Yeah, now the question is how sticky is this?

There is one question here from Enrique. By the way, thanks for all the audience questions here. Enrique asked about the CapEx interest, a CapEx story around Microsoft. Because if you see now all those services coming in, eight times higher AR, that is a huge shift away from OpEx over to CapEx.

We saw this in the market. But what are the risks traditional SaaS companies have now in terms of seat churn, price pressure, if you suddenly see that vast value differentiation happening? Is SaaS dead, essentially? Yeah, I think there are two different questions.

I don’t want to conflate them. The question number one is really about change and revenue recognition. And that’s a separate question from SaaS. SaaS is a value question.

And I think the reason we’re seeing this compression, multiples compression that has just occurred is some of that because of what is really happening. AI is the ultimate equalizer. It changes how we communicate with machines. Andrew Karpathy way, way ago now said, right, the new programming language, is it?

It’s English, but it’s not just programming language, right? It is a language of business is not going to be English. You no longer need SQL, right? You have a database.

You just ask it for what you want, right? And what is a SaaS product? SaaS product is predominantly two things. One is an interface access into the database, right?

An interface of the database. And that is the piece that’s getting really heard. Second one are the workflows, which is the business processes that are baked in. So the second one is currently still protected, but who knows, right?

If, you know, as the models get smarter and better, they will be able to at least generalize, you know, the processes that exist in a lot of industries. However, my bet is there’s going to be a lot of uniqueness still, but as you were saying, once the value is eroding and that space, so the SaaS companies need to really quickly, start to changing both changing their seed models and changing how they are delivering value. So if you talk about, you know, I sit on the board of Zendesk, for example, and then this very quickly scaled their AI revenue, but they were extremely brave. They took a step of pricing the product, not by seed, but by success rate, right?

So you only basically pay if the case is successfully resolved. But that is unheard of, right? It’s extremely disruptive. And that helped them drive the AI revenue at a much, much faster pace, faster than even the startups that you see.

Zendesk is an extremely good example, I think, because Zendesk for the folks who don’t know Zendesk, they are doing a customer service bots, or they used to do customer service bots. And that’s super boring. This is kind of like bot A, answers bot B, like linear, but they are doing a lot of linear workflow. Now, the first thing then they did is they started to acquire Ultimate in order, like a company which did more service bots and a large language model set up.

So to actually still have a linear workflow, but that linear workflow with more human interface. And then they invested into parallel workflows so that you start now being able to have a conversation. Now, the thing that changes how a product manager actually works, Lila, you have seen products develop, you have seen AI strategy, you have seen many product managers. What’s the future of product management?

I actually think product management is going to, we’re going to see a switch, right? That we’re going to need more product managers that are very, very technical. Sort of think of it as a, almost at Microsoft we used to have technical product managers, right? These are fundamentally engineered, right?

These are fundamentally engineered, engineers who understand user experience, design, business, right? It’s kind of all in one. And they can really, you know, outline how software has to behave, right? At a very detailed, in a very detailed way, but in English, you know?

And that skill is going to be really important to work with the models right now. You know, at least until, you know, the models start designing well themselves. I think that is where even engineers are going to transition. A lot of engineers are already transitioning to that.

Because as an engineer, you specify what you want to build to the model, and the model spits out a bunch of code. So I think that, you know, we used to say, okay, one product manager to 10 engineers. This could reverse in the next, you know, five years, you know? Yeah, yeah.

I very much agree. I think the ability to code is a very specific, way of describing problems. We are using now the human language, which is way more ambiguous, to be able to be defined and specific so that the computer can do what you want to do, is becoming a new skill from a product manager. I also think, and this is the Zendesk story, to be able to understand a business problem of your customer and cut it down into specific entities so that the agents, the AI agents, can work it through, becomes a new PM skill.

There is a saying about the IP is no longer in the SaaS interface. The IP is in the workflow you describe as a PM. It’s, you know, there’s an old proverb, right? It’s better to have a good question, the strong question and the weak answer, right?

And this is the job of a good PM, is to formulate very good requests for the LLM. And we see this, we have an incredible company in the portfolio called Factory, right? They’re actually building the entire ecosystem around construction of software, right? And the reason you need that is because building software isn’t just writing a little bit of code.

It’s the whole process of building the software and defining it and creating, by the way, the test-driven development finally is going to exist for real, right? Because the models are going to require it. You need verifiability. So you need to build the test on this before you build software, right?

Continuous integration is going to be an absolute, you know, no-brainer, right? We invested in that, right? We have a company called Namespace, which is specific for AI continuous integration and development. So the entirety of the stack will change and the jobs will change as well, congruent to that.

That is very nicely said. You will. You will. The IP is going to be the best.

The IP is in the workflow, meaning no user interface discussions, but workflow discussions. The quality is in, like, the knowledge is in the quality with continuous integration, right? Meaning it’s a quality and control question, again, having evals, And then thirdly, being able to communicate to machine, even when that becomes easier and easier, it still is important that the human language needs to be specific enough. It still is important that the human language needs to be specific enough.

Well, somebody has to decide what’s going to be good. Maybe one day machines will do that part too, but so far we’re not there yet. No, last question, and this is because you touched on it early on, and I didn’t want to go down the rabbit hole, but we have two more minutes. So let’s see.

You talked about AGI. Now, fair warning, I always said, like, I don’t see AGI anywhere. I just recently said it as I asked ChatGPT. Should I take my car to a car wash or should I walk?

And ChatGPT said, well, if it’s closed, then walk, which I still have a dirty car now. But anyhow, where are we in AGI? When will it happen? When will it happen?

What’s your guess? I think AGI is a massive misnomer. We’re dealing with intelligence that’s not like our own. We’re just trying to say that at any human task, you know, the system is going to be better.

I mean, that’s a very subjective thing, right? Is it going to be better at hugging me, you know, than my child? Probably not. You know, so I think it’s just a complete myth.

But what we are building is an intelligence that in many ways is very much superior to ours. And that is coming in some ways. Look, in many ways, it’s already here, right? You know, the models that I work with know way more than I can.

I can never read in a lifetime. So from that perspective, it’s here. It’s with us already. That intelligence is we’re working with them, you know, day in and day out.

In terms of, you know, singularity, I think you have to just define it for yourself. But we’re definitely giving birth to intelligence that’s much bigger than what we can fit in our brain. That’s very well said. Lila, thank you so much.

Thank you. Thank you so much. Ladies and gentlemen, thanks for, like, you saw an amazing hour where we touched on a lot of topics going from the future of AI, the last models over to how VCs are using it and how product management is changing. So it was a full hour.

Lila, thank you so much. Thank you.