Marc Riera7:03
It's based on predictions. Since working with words is complicated, these models tokenize words, converting each word into a number. Then you have a dictionary of words to tokens. With numbers, you can make predictions. You give me an input text, I convert it to numbers and compare with a database of numbers. It predicts what comes next. Models like ChatGPT have an extra training layer to act as a chatbot. Otherwise they are autocomplete. For example, if you say 'Hello, how are', the most probable next word is 'you'. The models compare what you entered with what could come next. OpenAI added a chat layer, converting an autocomplete model into a chatbot that predicts based on your request. If you ask 'Who is the president of Mexico?', it looks for data related to 'president' and 'Mexico'. It's a prediction within natural language. It works, it's intelligent and predictive. Sometimes it has more or less context depending on data quantity and curation. That's basically it.
You're familiar with the Turing test, right? And there's John Searle's Chinese Room argument. To quickly explain: The Turing test says if you communicate with an entity and don't realize it's a machine, it's intelligent. The Chinese Room is a thought experiment: suppose I put you in a room with Chinese symbols and give you instructions to respond to certain symbols with others. People outside might think you understand Chinese, but you don't. So my question is: Where do you think we stand? One can have almost metaphysical conversations with ChatGPT if you take the time. I think the Turing test, as originally conceived, made sense for previous technology. With ChatGPT, the method should be different. We get into murky waters: Is something that perfectly imitates intelligence but isn't intelligent? If a system behaves exactly like a person but doesn't understand, do we call it intelligent or an automaton? I think we need to compare neural studies in the brain with these computer models. My personal opinion is that we are talking to stochastic parrots. But what may differentiate a model that feels human is the critical amount of data. That's what happened with the generations: from first to second was a quantum leap, and the third has been universal. Now these tools are becoming interesting for all humanity. I'm practically convinced we haven't created life, but we may need to redefine concepts of intelligence.