There are many, many use cases which we should actually point out to. And we can’t talk about why autonomous agents, it’s probably a good hypebuster here. So you have the autonomous agents who are doing stuff for you. They break down tasks and then they’re doing stuff for you.
Good morning, Lutz. Good afternoon. It’s morning here. How’s your coffee?
My coffee is good. I have a data-driven, AI-driven coffee. That’s the main difference between us two, right? Your coffee machine is intelligent and my coffee machine has buttons and one light telling me it’s not hot anymore.
My coffee machine has about 10 sensors and every sensor uses an AI to actually forecast what the sensor really should do. I mean, we always said AI and data is for nothing if you can’t do something with it. Here, you get a good coffee. It’s always the same taste, an amazing coffee.
Yeah. So talking about AI, we already spoke about a couple of hypes, a lot of attention to AI now, and it feels a bit like there is another hype in the AI sector and that hype is around AI agents. Would you agree, Lutz? Kind of.
I mean, we touched on this actually last time, right? Because what we said is it’s an interface. And we said now we need to have different ways of working with that interface. So we’re talking about a large language model, but large language model essentially offer an ability to be an interface.
And as an interface, this is awesome. But now what to do with this interface, right? It’s the same saying as like, oh, I have mobile. Yeah.
What to do with mobile? And we show ample of evidence that large language model can’t calculate well. Well, large language model can’t access the internet. Last time we touched on a little bit on it that you will start now putting those abilities together.
Essentially, you give your calculator to your large language model and said, instead of typing on this calculator, I want you to press the buttons. And that is workflow optimization. And that’s what’s happening more and more. The coolest thing from OpenAI was actually the idea of doing plugins.
And this is kind of exploding at the moment. And if we look at it, I mean, we used Midjourney to create the picture for our podcast. And we have various pictures. And we did a lot of.
And participants suggested that participants suggested that participants suggested that participants suggested that participants suggested that participants suggested that GitHub stars. And yes, we can debate what a GitHub star means. It’s actually a symbol of KPI of quality. But this is quite a fast application of something on GitHub, I assume.
Let’s dig into it. So underlying auto GPT, and there are, by the way, many. So we have baby AGI, artificial general intelligence. And it’s actually a sweet, like this is where everybody is scared of.
They’re saying, Oh, my God, artificial. That’s why they call it baby AGI. So it’s a baby God, right? Don’t worry.
It’s just a baby. Yes. It’s like baby Yoda. And now it’s a baby AGI.
The G stands for general. And a long time we said, Oh, this is so cool. You have built a machine which is better in chess than anybody else in this world. Oh, so cool.
Now you have somebody who can play soccer. And every time we did those things, the answer was, but it’s not general. That machine cannot think about a poem. That machine can’t have an idea, and so on and so forth.
Now we have a large language model, which seems to signal to us emotion. But now the question is, is that general? Because it cannot play chess. And now baby AGI means that’s cool.
But I could actually link that interface, the large language model, to a chess computer. And that’s what we’re talking about. And that’s why we’re talking about the AGI. So now I have actually a more humanized interface to a chess computer, to a Go computer, to a mathematical calculator, and so on and so forth.
And that’s the reason why the G suddenly came in. But out of GPT, we have baby AGI, we have Camel, we have agent GPT. At the end of the day, I mean, this is not new, right? We’re talking about automation here, you mentioned it.
So we have a hype, or let’s say even amazing companies built around RPA, robotic process automation, which is basically scripting, I create rules, and there is not much deviation from those rules, except for a bit of human input. But to your point, that’s not artificial general intelligence, that’s just rules that the machine just follows. And now we spoke about prompting. So I can talk to this AI, I have an interface I can input, but I still have to give various trials of the output, and then I have to do the follow up.
So this is different, right? It’s auto. It is. This is different.
And this is actually pretty good in terms of how you described it. Because once we had a structural approach, this structure is human made. And actually, in the whole discussion on AI, we went through different waves of excitement. And those different waves were always based on a structure, meaning the first programming effort, or like having a machine to do something without being explicitly programmed, start the first wave of AI.
This still follows the human structure. Then we released that human structure and let the machine figure out its decision trees, whatsoever. And in order to connect those machines, we had to have an API. You actually describe this very nicely.
Because an API, or a workflow is a human-made, very clear structure. You actually describe this very nicely. Because an API, or a workflow is a human-made, very clear structure. structure, but we humans talk not in APIs.
And in order to have an API, application programming interface, you actually have to have an engineer to design that. And it can only follow certain rules and it can only do certain things. API basically means I don’t have to log into SAP or any other software program. I can use my software that connected to that software and kind of remote control access the other interface.
We can do this with the calculator, right? So if I ask you, Jasper, calculate two plus two, then I tell you this and you know that you need to press two, press the plus button, press two, press equal. Now, if I do this with an API, depending on how the computer interface from Jasper used to be, I would need to say, hi Jasper, I’m Lutz. This is my API key.
I want to do a calculation. Press two, press plus, press two, plus, equal, read out the text, send it back to me. That’s an API call. And it might be more or less easier or complicated depending how you design it.
And what we now say is actually, you know what? We don’t need those API calls anymore because we use our human language. And that has big implications because we saw it used to be computers have APIs or applications. And we used to have a lot of software that was actually using programming interfaces.
And they were complicated. Then we used the first wave was actually to free them by making it easily accessible. And then you had Zapier or if then, then that. And now we actually saying, don’t worry about API calls, just use a human language.
And we have the language model interacting with everything else. And the crazy thing about it is, and most people probably don’t know it, these integrations into other software. But if you have a machine that can actually do it, and it can actually work through APIs, that’s kind of a tedious work, a very intensive work, or at least time consuming. And tweeted yesterday, Kent Beck, one of programming legends said the value of 90% of my skills just dropped to $0.
The leverage of the remaining 10% went up 1000x. So this guy seems to love what is happening right now. Absolutely. Because you think about it, if I can have a machine doing those steps in between, it’s awesome.
But I don’t want to oversimplify all what’s happening here with AutoGPT. And by the way, the underlying discussion for it is actually long chain, language model chaining. So meaning take a language model as the interface and chain other models to it. Open source framework, right?
It’s an open source framework. It’s amazing. And it has two areas. One is, as we discussed, chain the API line, use the language as a connector, which is cool because it opens up for us.
Easy connection as did if then, then that or Zapier or any of the other tools. The second one, which makes it so important is if you look at that large language model is a finite machine. And we talked about this last time. It means it only knows certain things.
If at one point in time, I make the joke that I wanted it to summarize my course at Cornell, I would have to do it. And obviously, chat GPT would not have known in all the data what my course is because my course is not that important. However, I prompted it. I gave chat GPT the information saying, this is my course, by the way.
And now later on in the same session, I ask, can you summarize my course? And obviously it remembers because it’s in the same session and summarizes as well. But it’s in the same session. It might next time.
I talked to chat GPT. It will have forgotten it. So what’s important here is that we are now connecting big memory, big database. We talked about the importance of data and everybody brings their own data.
This scales up chat GPT big time because it now does everything based on your learning and your data. But the amazing, I mean, coming back to productivity, and this is what we’re waiting for, right? It’s a nice consumer application so far. It took off very fast.
And that has suggested that participants participants that participants participants participants participants participants participants participants participants participants participants participants participants participants I just type it in in a chat and then maybe books a flight for me or search in the internet and buys me some shoes on Zalando. Totally. The question is, how do you break down the task? If the task is press the buttons on a calculator, then somebody will make this connection.
Yeah. And if we are looking into the space of good investments, the question is, we said this last time, there needs to be this ecosystem upcoming. And Longchain offers the open source platform for this. So now it’s a race on who connects the coolest and most widely used tools for those large language models and who helps me with prompting.
There are two issues to overcome to make this business workable. Issue number one, connect the tools. We said an API call is difficult. Now we have a language model, but it still needs to be connected to the actual calculators that we can.
That’s number one. And it hallucinates also when it connects? I mean, that would be a problem. And it might hallucinate, right?
So now the second issue is actually how to do the prompting. I mentioned last time in mid-journey, the very fact that it’s complicated to make a prompt. If I train a memory to keep the prompt, which I like the most, you know, Starbucks said like, okay, if you go in and you say, I want a rice ristretto, 10 shot venti with breville, five pumps, you know, that’s a good idea. And then you say, I want a seven pump vanilla and seven pump caramel, splendor in it, a poured, not shaken.
Then you have a very specific view on your coffee. And nevertheless, it’s a complicated order because also Starbucks is an API. Essentially, you have to say it in the right order in order to get the fastest response. Now, an LLM takes any order because they know yesterday’s order and the day’s order before.
And Starbucks actually initially said, our employees know you. Once you come in, you just need to say same order as yesterday, but a little bit more vanilla, please. And they will remember. So that is the humanized version of an LLM plus memory.
I really like that. I mean, it’s trying to figure out what I want with my input, with my prompt. And so far, as we discussed, it doesn’t remember. So it’s not really progressing.
But this promise, and maybe you can explain this a bit more technically. The promise is I tell. The auto GPT or whatever camel. This is what I want.
This is the task. Please come back to me when you have a solution. So it seems to be even iterating until it solves it. It sounds a bit like reinforcement learning for all the listeners who don’t know what that is.
But it’s different than supervised learning where I basically say this is what the data means. This is the output. So when I put in new data, this is the likely output that you should predict. Reinforcement learning tells me, no, this is kind of when you look at a game.
You should come to the end without dying. So how does the AI know what success looks like here? It doesn’t. So most basic forms, we have those two areas.
We have connecting different tools, connecting your calculator, connecting the internet, connecting an order engine, connecting whatever, as well as the memory to keep understanding what’s happening. Therefore, we see the biggest success actually in all the tasks. So we have these two areas. We have the data.
And then we have participants participants participants participants participants participants participants participants participants participants participants participants participants wrong here, chat GPT critiques that changes the next step is please rewrite the code, including the critique back into it. So you have an iterative loop till it actually works. Meaning the combination between knowing what I’ve put in, in the past memory, plus the ability to talk to compiler, plus the iteration workflow, that ability created a loop to create good code. And it’s more difficult if the task as such is actually not very clear.
So what I did yesterday night, I asked baby AGI, baby artificial general intelligence, what everybody scares the heck out of it, and it doesn’t exist yet. But it is general because it might ask different tools. Now it doesn’t in my case. I just asked convinced venture capitalist, Jeremy Zee, to invest into me.
And my new AI startup. And I told it do four iterations. Now the way it doesn’t know, okay, let’s say, okay, in what I need, I need a task list. Okay, so my first task is research past investments, create a business plan, and so on and so forth.
And our listeners will have done this many times. And then the second iteration is okay. Now for business plan, actually develop a marketing strategy. And let’s for that.
The next iteration is in order to do my strategy, I involve multi channel marketing, la dee da dee da. It kind of breaks it down step by step. But that does it in iterations. But what what is confusing to me now, I could write this from the start, right?
If I look at business plans in the internet, yeah, I have to do all of that. Why does it need iterations for it? Because you’re human. And you’re a multi level complex here.
First of all, there is no. For this baby, a GI, there is no G actually, because it’s not connected to anything else, right? So it is only doing the iterations. Yeah.
And when we ask GPT or BERT or any other tools out there to solve a problem, the more accurate we actually guided the tool with prompts in order to saying do first this and then that the better the outcome was when we ask a mathematical question, it might hallucinate. If we said, actually, it’s a mathematical question, I want you to actually do a calculation. The question is the following, then it worked better. So essentially, by using that loop, we are asking to iterate.
I have a very personal message to all the listeners. Don’t write your business plans with looks a GI. Please, please use still use your your own imagination and research. Dear listeners, don’t listen to me.
I’m not a Jasper. No, but I actually think there is something to it. The world will be scaled up in many respects. So to to follow a certain structure, to understand the structure is something where we can use AI or large language models to break down tasks.
We can use them now to fill them and to make them useful is obviously something where our personal innovation. We saw this in the music industry where somebody makes an artificial intelligence song from Drake and it sounded very real. And the whole point is the style of Drake is what the world loves. And that is what the computer imitated.
I would like to come back to a practical example, which could be a nice segue into the limitations of this, at least right now. This was you about a business plan. And we got this example in the E-Learning. Yes.
I have to read the name of Robert Jesus. What he did was he used AutoGPT to order a pepperoni and sausage pizza from a one of those pizza places in the US. And it was mind blowing. Only took one hour and a thousand dollars of open AI credits.
So that’s a very expensive pizza. What is the issue there? Why is it iterating so much? Why does it have to use so many open AI credits?
What’s your interpretation? I mean, obviously, we haven’t seen it in detail. We don’t know. I haven’t seen it.
But I think it’s a good idea. But I actually think whatever he did, he did it wrong. It shouldn’t have cost a thousand dollars. But next time, give the thousand dollars to me.
I’ll make you a pizza. That’s fine. Maybe talking about the applications. Yeah.
If you go to the website from Longchain, the open source framework for this, while at the moment, all the hype in the general media is around agents, autonomous agents. There are many, many users. There are many business cases, which we should actually point out to. And we can’t talk about why the hype is autonomous agents.
That’s probably a good hype buster here. So you have the autonomous agents who are doing stuff for you. They break down tasks and then they’re doing stuff for you. Like business cases could be question answering over a database.
Right? Remember now memory database question answering is language interface. Go to database, figure out the information, distill the information and report it back to me. Change the database.
If you want to have a chatbot interacting with a booking system, then you have language interface, needs to understand how to access the booking system. Again, that’s a long chain and come back. And it has to remember my feedback, right? And has to remember my personal style coming back to your coffee ordering example.
If you take Harry Potter, all seven books, try to copy paste it into OpenAI. You run out of time. Yeah. And you have to remember that you have to have a lot of time.
It’s a finite machine. It cannot keep everything up in your brain. What you do is summarize page by page. Then you store that in memory saying, now do the summarization and you can do an iterative process and you will get your summarization.
I think it’s about wizards, but that’s it. Right? So all of those are amazing use cases. And the days are counted for.
A person who just took information from one side of the desk and had to tell it to the other side of the desk. The days of pundits like us is counted. We will be replaced. But not yet.
Because if we look at the issues and we tapped upon it at the beginning of the podcast, but also in previous ones, the agent might still hallucinate. So the model might make up things and then it books a flight to Dubai and not to Singapore. It might even take decisions that… I don’t want.
So I would still probably need some feedback loop. Right? So the models are not there yet. Is the solution asking humans more often?
So this kind of auto promise basically means I have more feedback loops with the agent and I don’t have to prompt actively myself. Or what’s the solution? Yeah, I think the issue is actually that human language is less accurate than computer code. And therefore we will not…
Mm-hmm. Just completely replace an API by having a human language. Now, the human language will allow us to actually ask back, okay, if you want the number of patients in a certain given state split by gender or just the number of patients, kind of have that dialogue, which will help. The AI doesn’t know what to ask until you tell.
And I also read these AI agents, they tend to forget. So it’s a very good point. Like, if you have a certain task, so it starts with something and then in between it tends to forget, is that something that will be solved with technology over time? That’s a finite machine problem, right?
So if I have then the memory, it actually should play it back into it. Now, I want to come back to something else you said earlier in terms of the self-training on API. Because we could use that long chain model to improve data quality. Meaning when we loop, and you talked about this early on, then how to train.
When we loop over data over and over and over again, we train the model with something. Yeah. Now, we have seen this in the past, a study done by Goodfellow in 2014, where we came up with the GAN, architecture GAN stands for General Versailles Networks. And that was used for image generations.
Mm-hmm. So the idea was… You have a computer generating an image, and you have a computer trying to figure out, is that image fake or real? Meaning those two computers are adversarial.
They’re working against each other. One tries to cheat, the other tries to figure it out. And they would go in a loop and do it over and over and over again. And I heard some people might have misused that also to create some fake pictures and videos.
But yeah. Totally. Actually, most of the… I mean, I have a whole Forbes article about this, using more placements like the Snapchat style, putting something on somebody’s face.
You can do this actually with real pictures as well. But the whole idea here is train the computer by itself. Tell a computer to play Doom and don’t die. So the computer will try, and it dies.
And then it tries again, and it dies. And it tries again, and it dies. At the end, computers are painless. They can try it over and over again.
So the listeners, that shows you we’re a bit older. So Doom is a game from the 90s. Nowadays, it’s called Fortnite. The point in all of the deep learning is you need a lot of data.
LLMs are trained on a lot of data. We figured out with the GAN architecture, the way to use data over and over again. Now with Longchain, the cool thing is we figured out a way. We figured out a way to actually train computer by itself.
Meaning because you have memory, where you store and you have an interface, which might be only your own interface. And you go back and you train over and over and over again, till you get it right. In order to do this, you need to know what is right. I was about to ask because I mean, I can imagine if infinite loops are this going totally sideways.
The model just doesn’t know where to go. Yes, totally. Like for the task, how do I get money from CherryVC? I cannot try this a million times.
Because you don’t have a result, right? You don’t have a result. The amazing part, and this is where we will see in the future a lot of development is whenever you get a feedback loop. So that’s a reason why AlphaGo, the computer who plays Go, they got trained on existing data.
And then you move to AlphaMuZero where you said, I just give them rules and I let the computer play against itself and it figures it out. We did it in chess. There’s always clear rules and you have a winning and do it in Go. You can do it in programming because there’s a clear rule when code is executable or not.
So now the computer comes up with new ways, which people might have not seen before. And why? Because it can test it so many times. I really want to make sure.
I create a good product out of this if I use it for whatever application is out there that I think I can now finally automate. I think a very easy one is you said this agent has access to data, could be personal data. So I guess privacy from my European perspective is an issue. So I have to make sure that whatever the agent is doing there, data privacy ensured and still it can use it.
Is that a challenge? It’s a challenge because the question is. What type of large language model do you use? As soon as the large language model uses your data in the cloud, no matter whether your connected device and your connected storage is on your side, moves into the general knowledge about the large language model and we will see changes there.
I’m excited to see a generic platform to emerge, which connects many useful tools and everybody’s going to connect. I’m excited to see more. New participants suggested that participants suggested that participants suggested that participants suggested that participants suggested that participants to me, yeah, it could also be a threat for Zapier itself because now I can connect everything, right? Totally.
The other big, amazing thing I think we will see is that we have the data question. It still comes down to the data question, like what type of data do we use? So I’m not super excited about auto GPT and autonomous bots for now because they don’t have enough data. However, I am excited about companies like Ultimate or others who have a data set.
They know how they want to interact and work with a situation. They have the connection. And now they need to chain this together, long chain, to change this together in order to answer questions, in order to get an interaction done with an API or in order to get summarization. Those are the use cases where I think there will be.
A lot of amazing new developments happening. The third part I’m actually very interested in is if we can train the machine to train itself to always write correct code, then we don’t need a coding language anymore. And I guess also the question is, where do you start? I mean, we spoke about quality controls.
There’s guardrails, AI. There are other also frameworks how to control the output. So hallucination is not happening. But as you said, when I start my company, when I build.
An AI product, how do I make sure the initial output is good enough? The feedback, I feedback is good enough. And then over time it becomes better and better, which is kind of a mode because I did it and others maybe failed there as you described. So it’s a really a combination of the user experience, but also knowing what kind of quality I’m getting, which probably is due to the data I’m putting in.
And I’m also knowing a little bit the output, which is kind of back to supervised learning again, or the basics of how to. Build AI products. I wouldn’t call it supervised learning. It’s more than that, but it’s down to the basics.
You need to answer the right question. You need to answer, like create the value. You need to control the data. You need to have a workflow to operationalize and update your model.
Yes, absolutely. All right. Thank you so much, Lutz. I think you deserve another coffee now.
This is awesome. I get another coffee. This is good. And I’m going for barbecue now because it’s evening and I’m hungry.
Wish you all the best. Have fun. Enjoy.