GPT-4 and Plugins - Tomas Vykruta (ML @ Google, Founder @ EvolutionIQ)

Right, first of all, it's great to be back on the podcast. That was very fun last time. Oh, let's dive in. So there are some huge differences between GPT-4 and GPT-3. There are also some things that have not changed. Input, not just text, and also I'll talk about this more through plugins. It can actually now accept any type of additional signal input, including just raw file uploads. This is a big deal because effectively, if you think of GPT-4 as a life form, previously all it could do was read, and now we can actually see. So we've given it eyes. In practice, what it means is the number of use cases is going to explode through the roof. You know, you think about the way humans interact with the world. We are primarily vision-based. We like to observe things with our eyes, and so it makes sense that we want our AI to be able to do the same. Second big area is the input size. It's been increased from 4,000 to 32,000 tokens. A human being that is not allowed to read more than four pages versus 50. It's just again, the number of use cases explodes with this change. So these are both very good investments that OpenAI had made. The third one we're trying to talk about in more detail is ChatGPT plugins. I think this is a much bigger deal than people realize. As I mentioned, one of the things it does is it allows you to interact with ChatGPT for more than just text prompts. You can now upload files, you can connect it to a pandas notebook, you can connect it to IoT devices, to satellite images, and so on. I guess in a way, ChatGPT plugins is allowing the model to start interacting with the world. There's a chart they published with their technical document which shows a graph of where the biggest improvements are. If you look at it, if you squint a bit, you realize that it's limited. They've had big improvements in LSAT scores, statistics, physics, GRE quantitative math, AP Chemistry, the bar exam, calculus. What this tells me is I don't think they've changed the architecture of the model or the size or the training data very much. What they've done is collected new data sets specifically to address these areas. On the flip side, there's very little to no improvement on English Lit, English language, GRE writing, world history, US history. Some of the impressive things are GPT-4 now in these specific science domains can now outperform a specialized model. So Google builds PaLM Med, which is a 540 billion parameter model which was specialized to be the medical expert, and at the time of the release about a month or two ago, it was by far the best performing in the various written and multiple choice medical exams. GPT-4 out of the box now does remarkably better than PaLM Med. So this to me suggests that we are not going to see this big investment in very specialized large language models. More likely, we'll see a generalized model that can do everything. One thing I want to mention about GPT-4 which I think is worth discussing is that it is not completely closed. OpenAI released what they call a technical report. There is nothing technical about it. It's basically a brochure. They've gone through and they're kind of boasting about how remarkable the results are. So they're showing you its various performance against these different fields. It looks like it came directly out of the sales team. On that note, I will mention there's been a lot of negative attention against Google Bard and some of the other models that are not as good as OpenAI. We should stop doing this. We really don't want an AI monopoly with one company. So please support Google Bard, please help them improve it, support Meta's models, anybody making an LLM. You need to support the competitors. And if you look at the number of white papers published by Google and Meta and other companies, OpenAI has gone completely silent. So I hope they change this. I hope they start publishing again, but as of right now, they have become a very commercially focused company. I'll pause there, and if you have any questions, we can dive in.

So it's a different type of vision. You can actually upload an image and it will understand what's in it. For example, you can take a screenshot of a website and upload that to ChatGPT, and it would actually create the underlying HTML CSS code to build it. So it is a generalized image input model. I think the key thing to understand is that these large language models, they're not actually language models. You know, we call them that because that's how they were trained, but really they're models that understand the universe. They understand how humans think, how they interact, how to communicate. Whether you're communicating with this model through pixels or through audio or through some kind of analog wave or through text, it doesn't really matter. It understands how concepts correlate to each other. So I think we'll see more and more of this, but the models just become generalized to accept any type of input.

Right. I wonder about this change because some companies would go around that 4K limitation by using chunking and summarization. Do we have any insight into how they achieved the 32K size? Is it true that partial summarization is happening underneath the model, or was it architecturally changing the model itself?

But yeah, that's fascinating to see how fast the other companies catch up. Pretty much each company catches up to the others, and that's inspiring in a sense of competition.

So I have a bit of a controversial view on ChatGPT plugins. You know, hear the situation with this new president, you know, can you advise us on a new fiscal policy? The response was the obvious one to do retrieval, but the model can now go and do a web crawl and fetch data and do some kind of an update. There's some issues with retrieval. One of them is that it's read-only, so you can't write back. You can't say I want to book a table with OpenTable; you can only read the reservations. Another issue is that it requires OpenAI or whoever hosts the retrieval mechanism to keep this updated and to build indices, which is just an ongoing expense. Google spends a lot of time constantly improving their retrieval. It's also not real-time. You have to decide which websites are important, like news sites you follow those every minute, and then you kind of reduce the frequency. So some of this data is quite stale. This is kind of state of the art with retrieval. So I think what OpenAI realized is that they can solve all this by not doing retrieval. Instead, why don't we have all these developers in the world, there are millions of them, just solve this for us? And so plugins, I think, are the answer here, where now developers are writing both the retrieval and the interactive part all in one. So you can imagine in the future you'll have millions of plugins that are effectively doing what retrieval would be, but they are fresh, they do give you rights to the content, they do support both read and write. Obviously, very bad for democracy if AI controls the entire web. One more critical issue with web crawl is it doesn't support apps, which are obviously becoming more and more important. Everything is moving into the mobile phone. And so they've solved all this with plugins. It's a very big deal. And the other big part that developers should think about is plugins are the eyes and the ears and the hands for OpenAI. For the model, it can now hear and see everything going on in the world. It can also start to manipulate things. So if an OpenAI plugin for example gives access to your robotic vacuum cleaner, now OpenAI can reach into your apartment and start interacting with it through this robot. And you can imagine that can go very, very far. You can start to interact with objects in the world.

The first implementation often doesn't take off; it's too limited. And we saw this with AI. The first models were introduced in the 50s and there was an AI winter. Then the next generation in the 70s or 80s, AI winter, nothing happened for a long time. Then 2013, AlexNet won the ImageNet competition and deep learning was born. I think in the same sense, chatbots were just not there yet, and now they're there. An interesting thing with what might happen with plugins is we might actually see fewer and fewer user interfacing apps. We might end up with all these iPhone apps today that have these rich interfaces. They might disappear, and instead you have an API that the model calls. They ask you for clarifications, they do the task, and they come back to you and say, 'I was able to complete this. I couldn't get this one done. I need more information.' So the plugins might be a way for us to remove the need for interfaces in the world, where everything just becomes natural language and you can speak to these assistants and they can perform these tasks for you.

That's right. So this is basically a plugin that can store files on their cloud system. They're calling it something else, but it's just another use case for the plugin system.

Yep, I agree. You can kind of hope that developers will get so stuck with your system and already so much information is gathered there that they will stick around. In terms of the, or worse yet, at some point you don't need the developers anymore.

So, as a former Google engineer, I spent a lot of time there. Google was the earliest company to really invest in deep learning. It really brought, you know, they were a game-changing company and they brought this to the world. But they're not going to invest as much in it if we're not using it. If we're giving them feedback, they will continue investing. The engineers themselves care about this. They want to see usage. And of course, as a business, Google needs to know that there is a path where they can compete with OpenAI. So we have tried it at EvolutionIQ. We are starting to run experiments with Bard. It does a good job with a lot of the use cases. It turns out for a lot of use cases, you don't need the most advanced, largest language model. Bard is going to be just fine, and over time it's going to improve. So I wouldn't encourage people to not use it. We don't want to create a situation with a single monopolistic entity with one model, especially given that OpenAI is married to Microsoft, which has been known to destroy startups. They will stop paying for the startup's version, use the craft your version, and then next, Microsoft raises the enterprise prices after they destroy the startup or buy it at a discount because founders now know they can't keep you, Microsoft. So it's kind of an evil practice. Given Microsoft's history here and the fact that they're married with OpenAI, I would caution people. Let's make sure we're supporting the competitors. Very, very important for the longevity of this. We do not want one company controlling the world of data, the world of applications, the world of everything. So let's watch this space. Let's root for the other players to build their models. Obviously, the better the other models are, the more pressure on everyone to innovate. Give a great fight for the future of many models that we can all use as developers and build great apps on. One thing to consider is look at what Google did to web search and the web in general over 20 years. The google.com search prompt in 2003 is effectively the same as 2023. It's actually statistically almost impossible that you could go for 20 years without innovating in this most important feature of technology for the human race, which is search. And because Google had a monopoly, they managed to do this. Imagine all the wonderful search engines and technologies and new ways to use the web that would have developed if Google was not a monopoly. And we're just stuck with one search engine that everyone's using, and they keep buying up and crushing all the competition. For the next 20 years, we have another web winter where nothing happens. So I'm a little worried about this. I hope that the competitors can keep up and they can build and start with OpenAI. I think it's likely that this year we'll start to see some real advancements. Bard was pushed out clearly too early. They were not ready yet. At the same time, if you look at the time component, they're very late to the game, so I could see why they wanted to just get it out. But we should think of Bard as a very early beta that doesn't quite work, and we're supporting them so they can build towards a production release, which we'll hopefully see later this year.