DevNTell - Running LLMs Locally – Tether's Paolo Ardoino Reveals Decentralized AI Platform QVAC

Thank you for having me, Aaron. I'm excited, too. My pleasure, my pleasure.

Sure. I'm Paolo Ardoino. I'm the CEO at Tether. Tether is mostly known as a company that created the first stablecoin in 2014, USDT. In the last 12 years of history, we participated in all the different cycles of growth of the crypto world. But today, USDT as our main product has exceeded all the expectations and became the digital dollar for all the emerging markets, developing countries. It's used by 570 million people around the world. It has been probably the biggest financial inclusion success story in the history of money. So, very excited about that. And definitely USDT taught us a lot in terms of how to use technology to liberate people, to create freedom, to give access to the people that never had access before. In the case of USDT, there are hundreds of millions of people, billions of people that don't have access to basic financial services. And so, at Tether, we thought how to use the learnings that we had from USDT and the stablecoin world to also help the billions of people that will not have access to good but basic AI tools in the future, how we can serve them through our network, how we can make sure that there will be an intelligence inclusion, not just financial inclusion. So, that's the reason behind Qubic to start with.

So, I've been a geek nerd hacker since I was eight. Well, I started coding when I was 8 years old. My father and mother, they were busy sustaining the family and they were working very hard and I was very lucky because my father brought home a computer and I had a lot of time to play and then the games were very expensive. I come from a very small town in Italy and we didn't have much access to anything else so I wanted to create my own games and so I wanted to learn coding. And so I started that, I loved it immediately. For me it was a way to, many of my friends were good in arts and drawing, I was bad at everything and so coding was a way for me to create my own universe and invite people inside that universe. And so throughout the following years I started getting internet and I was playing with Linux. I started using Linux very early, well early for me was '95. And then started learning about hacking. I wrote some articles on e-zines, you know there was the hacking scene in Italy. I wrote some articles on RSA and large numbers factorization and some math already at a young age, very passionate in cryptography. And then went to the university and then became a researcher at the university. I learned and studied mostly parallel computing and distributed platforms. And you know, loved BitTorrent and what BitTorrent meant for the whole peer-to-peer space. I then worked and I think I led a very important project, we had amazing core researchers. And we designed communication systems, highly resilient communication systems for battlefields. You know, when you think about, and I think that thing shaped my mind. You know, whenever I still today design anything in technology, I always think how technology should be designed to support humans in their worst moments, not in the best moments. I mean, it's easy to design technology that works perfectly when everything is fine. But technology should be the thing that helps us humans in the moments when everything goes to hell, basically. And so when we were designing these solutions and the communication networks for battlefields or distress situations, I always think about fathers that needed to communicate to their colleagues, you know, when they were even risking their lives, right? So that makes you think more. That makes you sharper. There's much more at stake and so we designed amazing protocols and then I carried that experience throughout all my career. And I keep thinking about how again technology should resist to the wrath of God if needed, is truly designed to help humanity and society. And so yeah, I love coding, I love hacking and learning about things and learning how things are built.

Well, I'm a sci-fi enthusiast. My preferred author is called Isaac Asimov. He wrote hundreds of books. There are two books that are amazing. One is the cycle of Foundation and the other one is a short story called The Last Question. The Last Question is like 14 pages. Probably the most beautiful story in the world for me. In 14 pages it includes philosophy, science, physics, AI, religion, and a lot more. And you know, it's the story of humanity that from the 1900s onwards kept building better computers and better artificial intelligence. And you know, every hundred years the technology was better, the AI was better, then every thousand years it was becoming even better. And then millions of years pass and billions of years pass and this AI becomes always better. It's incredible. At the end, after billions of years this AI is part of the fabric of the universe. And throughout the entire history of this story, every single time, after hundreds of years, after thousands of years, every single time the AI was becoming better, humans were asking one single question to this AI. That is the most complex question of the universe. And is how entropy may be reversed. How we can basically stop the universe from dying. And I don't spoil the end of the story, but I thought what does it mean for us? And what does it mean for humans that, you know, we are part of the universe, but we still only know Earth. And so I thought, well, how can we transpose that question to Earth? It means how can we stop society from dying. How can we make society stable. And Tether happened to build a stablecoin. And that is a digital dollar. What we saw, and we analyzed in a very scientific way what USDT did and obtained from a sociologic point of view. If you look at the data, you see that USDT was able to bring stability to society. Society of the emerging markets, the developing countries. You know, people that live in Turkey, the Turkish lira lost 80% of its value against the US dollar last 5 years. The Argentinian peso 94% of its value against the US dollar last 5 years. So, not entering too much in USDT, but those populations living in those emerging markets could not be stable, could not be building a better society, a stronger society, you know, a society that is able to innovate and build and grow and study and research more because if your currency, if your national currency is basically destroyed every single year, the only thing you can try to do is survive. So, then there are not many people know about, there are 4 billion people that are affected by this and even more, probably around five, that live in countries with super high inflation. So, if we have basically more than half of the population of the world that is kept outside of the traditional financial system or does not have access to basic financial services in a stable currency, then how can society remain stable? So, going back to my point, how can we stop society from dying if half of the population, half of society is actually trying just to survive because of the currency that they hold is dying. And everything is priced in currency, right? So, how can we fix that? And so, USDT is the first example of something that truly can change the world. But, it's not enough. It was not enough. So, now we know that you have half of the population of the world that is part of the wealthy, maybe not wealthy, but still has access to good financial services and half does not. And so, there is a gap between these two halves of the population. And then, you add on top of that AI. So, what if AI would increase that gap? Because if half of the population of the world that has access to financial services plus has access to AI because they can pay for an Anthropic or OpenAI subscription. This half of the population will become 100 times smarter. And on the other side, you have the half of the population that does not have access to basic financial services and cannot afford to pay for an Anthropic or OpenAI subscription. The gap will become not twice bigger, but will become 100 times bigger because as people need stable currency, they need, in my opinion, stable intelligence. They need to be able to have access to basic intelligent tools that will help them throughout their life, that support them, and they will make them part of this digital revolution. But the problem is that if you have a person that lives in a city where the average salary per day is $1.35, you know, in many countries in Africa, you have the average salary of a person is $120 per month or $80 per month. How can these people afford to pay for a subscription to get basic AI services? That is not sustainable. And so, we thought that with Tether, we have already a huge distribution channel into those populations, into that part of society. What if we were to reuse that distribution channel to build open-source, local-first, on-device AI tools that would not cost a fortune to Tether to run, but would be good enough to solve 80 to 90% of the use cases of AI for that part of the population. So, many people in Africa or in Central South America and so on, they don't need, they will not use AI. They don't need, actually 99.9% of the population will not use AI to try to find a cure for cancer, for a very, you know, it's the most amazing thing that AI can do right? So it's very important the AI usage for solving the very complex problems of the world. That's amazing, but that's not what 99.9% of the people in this world will try to do. Most of the people will need simple things that will be very helpful to their day-to-day lives. Most of the people in the world have very important problems that are local to themselves. And they need, and if they use AI they will have a help. With that they will have a lifeline like using USDT will offer them a lifeline. And so why not using the learnings of USDT to build tools that we can offer to these populations that provide education, that provide basic health answers, that can provide, you know, we know that Tether is working well, today we released a model called Med-PaLM that is even beating Med-Gemini that is Google's model. And so our 4 billion parameters model is beating 27 billion parameters model from Google and can run fine on a smartphone, can scale on any small device even an average smartphone. So that is an incredible testament of what our team could do, but on the other side we are working on better translation models for African languages. In Africa there are many languages and dialects. So we are trying to put our resources to create the most amazing open source translation model for African languages so that suddenly we can offer free education to all the villages in Africa that we are already servicing with USDT. We're going to offer them free education so that all the kids in those villages can learn and can study and can be part of an amazing society. And so, these are the things that honestly don't cost even that much. But you need to have the right incentive to build. The right incentive to me and for that is stability of society.

Absolutely. So, Qubic, let's start from Qubic SDK, right? So, first of all, we didn't want just to build an AI platform on cloud. There's plenty. I don't think it's that exciting to us. So, we wanted to offer an SDK that would allow any developer to easily code inside their applications, existing applications or new applications, AI functionalities. AI functionalities that would vary from OCR to standard chat LLM to text-to-speech, speech-to-text, and so on. And so, we wanted to wrap everything, all these functionalities and more into one single SDK with one single interface with one single coding style. So, behind the scenes, we took llama.cpp that is an open source inference engine for large language models and we forked it, still maintaining it open source. And we call it Qubic Fabric. So Fabric is an enhanced version of llama.cpp that over time got a lot of additional support including for example the ability to run the Microsoft BitNet models. You know, the one-bit models that Microsoft pushed rather than the version of Microsoft that was only basically supporting Nvidia GPUs and the very high end GPUs. We adjusted so that now the BitNet models can be run on consumer hardware on the Snapdragon GPUs of Samsung phones or on iOS. And so on top of that we also allowed Fabric LM to support general fine tuning layers. So not only the BitNet models but all the models that are supported by Fabric LM that is basically all the llama.cpp models. They can now be fine tuned directly on local devices through one single framework. So we proved that with Fabric LM we could fine tune one and three billion models directly on an iPhone 16. So this is truly unprecedented. Over time, imagine like an iPhone 16 was possible but then you have your nice Mac or a good laptop. You can truly fine tune models locally with your own content without losing any privacy. So basically Qubic SDK is a layer on top of Fabric LM. Fabric LM is a C++ engine as a fork of llama.cpp but includes much more than llama.cpp and offers a unique unified API layer to do all the classic tasks that you want to delegate to AI: text chat, text to speech, speech to text, OCR and any sort of recognition, image generation. We support now image generation with diffusion, we support all these type of things. And so there was a huge reception from the community and we start seeing some traction from the open community to develop on top of Qubic SDK. We are going soon to run some hackathons to prove the quality of this tool. And so suddenly now every single, by the way this Qubic SDK can be used on, it has a JavaScript interface so that any web developer can suddenly start integrating. So web developers are the most common type of developers and so they can use basically Qubic SDK and build any logic inside any app in a very simple way. And this is our way to empower every developer to build AI tools by themselves maintaining the privacy but also reducing the latency at the minimum. So imagine a future where we are surrounded by billions of robots. I want a robot to be able to use a local model and local vision model and understand what is going on around itself and take a decision without waiting for the latency of going to a data center and back. You know, imagine like in a smart car, a car should brake immediately after seeing what's happening directly on the road rather than having to wait for that data center answer. And so all that, the world is moving towards local AI and we believe that every single developer should be empowered to control their own local AIs, and every single user will benefit from having a local private AI.