Hi everyone, my name is Steve Lee. I work at Block Inc. and lead the Spiral organization which Haley mentioned earlier. We work on open source Bitcoin and AI. And I'm joined today by David Sacks, who's the founder and partner of Craft Ventures, co-host of All-In Podcast, and also advisor to the president on a number of topics, but including AI policy, which is a hot topic that we're going to talk about today. David, thanks for taking the time to join us.
Yeah, thanks for having me.
And I think David, you can only see me right now, right?
Yeah, I can only see you, so I'm assuming there's an audience there.
Yeah, there's... Oh, wow. Okay, there we go. Hi, David. All right, cool. Yeah. And yeah, everyone want to say hi. And we, you're probably familiar with our venue. We're at the Golden Gate Club in the Presidio. So we also have a lovely view of the Golden Gate Bridge behind me. So...
Um again, thanks for joining us. So let's get started. I think a great place to start is the big topic from the past day or so and that started with an ex-Anthropic employee tweeting and now it's blown up into all mainstream media covering AI safety concerns. A double-digit percentage chance of killing all of us in a few years. And now it has like dozens of politicians commenting on it in all of mainstream media. So what's your interpretation of this from like a regulatory and political standpoint?
Well, let me tell you what I think about it, but first to say that I'm a big fan of open source. And you know, open way to AI. Because to me, open source is software freedom. I think it's really important that we have the ability to run our own AI models on our own hardware. To maintain control over our own data, you know, so-called AI sovereignty. I think that's really important. I think the biggest danger of AI is that it will enable a centralization of power and we end up in some sort of Orwellian world where you know we have basically a big tech duopoly of frontier AI companies that work hand in glove with the government and you know and have the power to surveil us, control information, censor us, all those kinds of things. And basically, you know, what happened with the social media wars just gets replayed with AI, but in a much more draconian way because AI is just so much more of a personal technology. I mean, AI is becoming to many people, it's becoming their doctor, their lawyer, their therapist. You turn to it for so many things and of course it's eating the web, you know, things that you used to do web searches on now you'll just ask AI and you can imagine a world in which you know the government again is working hand in glove with a couple of AI companies that have a monopoly or duopoly and that the kind of information control and surveillance that would be possible would vastly exceed what's been possible until now. So I think open source is just a really critical part of helping us avoid that potential future dystopia. Because again it would allow AI to remain decentralized and give people more choice, more freedom.
I think that so let me just want to state that because like those are my priors and that's where I'm coming from and as an advisor to the governmental policy that's what I've tried to represent and protect. And I'll just say that I think that what's happening right now is part of a larger campaign whose target is ultimately open source AI. It's not the only target. But you should just understand that open-source AI is the real target here. Because you know the economic interests and the political interests are both kind of unified in their opposition to stopping that decentralized scenario. I mean obviously if you're one of the two leading if you're part of the frontier AI duopoly you're you know you would like to basically you're pursuing a regulatory capture agenda that would allow you to kind of strengthen your moat and get you off this like hamster wheel of constantly having to stay ahead of open source right like that gets kind of exhausting if you're one of the leading companies and you have to stay ahead open. It'd be really nice if you could just derail all that by instituting new regulatory processes and reviews and things like that that open cannot comply with. And I'll talk about that in a second. And then the you know politically speaking again the people who favor centralized government control and regulation don't like the idea of AI again being this decentralized technology for the same reasons that I like it. They don't like it, right? It gets you know, if you have a control agenda, open is anathema. So, I think we should just understand that like this is an op. I mean, I can explain why it's an op if you want to go get into that. And the target is open source.
So, as I mentioned, dozens of politicians commented on this. I'm assuming it's a topic for them because they feel like their constituents are concerned about AI safety and worried about it. And I've seen data showing that public sentiment tends to be negative towards AI. So why do you think that's the case? What can be done to paint a more positive picture for the public?
The reason why public sentiment towards AI is so negative is because there's been a relentless information campaign to paint AI in a very negative way. And the way that you can see this is that actually they do opinion polls by country and it's not every country that has this negative impression of AI. If you go to like China for example, something like 80% of the population is optimistic about AI. And the way they measure this they asked the question Stanford did a study which is do you think AI will be more beneficial than harmful and something like you know they if you say yes you're an optimist and if you say no it'll be more harmful than you're a pessimist you're an AI pessimist so China I think it's something like 80 83% optimist. And then you know other countries in Asia you know South Korea Japan countries like that they all tend to be pretty optimistic as well. And then as you get into the western countries, it's more pessimistic and I think the US is the most pessimistic.
And I would just attribute that to the fact that billions of dollars have flowed into this so-called doomer community which you know kind of originates from the like this effective altruist rationalist school of thought that you know has mega donors behind it and they have created hundreds of organizations and to spread this message of doom and gloom and you know again if you repeat the message enough times and then of course you know if you can get people from inside the leading AI companies to spread that message it appears to be very credible. So that I mean that has shaped public opinion I think more than anything else. Yeah let me pause there.
Yeah. Well, I'm just curious who do you think can step up and help paint that picture, a positive picture and also it might require a lot of funding as well as you just mentioned. I mean it does seem like there's actors in the ecosystem who it's incentive aligned for them to have an open model and open source AI world. Do you think they'll step up?
Well, one thing I would say is that I do think that the open source community needs to form some sort of political group, some sort of you know, I don't know what the right structure is exactly, but and there are some groups, but I just think that the open source community needs to get politically active in a way it hasn't been because I can tell you that the opposite point of view is which is to say that AI should be that the safest path forward for AI is for frontier AI to be centralized in the hands of very few companies and to have you know approval processes that you know to basically create an FDA you know I call it an FDAI federal department of AI that would review and approve AI models. If you go back to Dario's testimony before the Senate I think it was in 2023 he explicitly said that open models if sufficiently advanced, which is to say, you know, have the potential to compete with him. That they are dangerous and specifically he said that they were dangerous because you cannot centrally monitor and control and roll back an open model, which is true, right? Once someone publishes the open weights into the public domain, then someone can just take that file and run it on their own hardware. You can't, you know, the genie's out of the bottle.
I think that those issues are manageable and those issues are outweighed by the positives. But in any event, that was the testimony and I think everyone should just understand that the regulatory approval process that's envisioned here in a new federal department. It will initially be claimed that, oh, we're not trying to ban open source. No one's trying to ban open source. We just want to make it safe. So that'll be the first thing they say. Then the next thing they say once they have a bunch of requirements is that these requirements must apply to both open and closed models. That's only fair. And then the third step is that you know some of those requirements will be things like you know models that pose a safety risk must be rolled back and they must be centrally monitored and controlled which again these are immutable characteristics of open weight models. And since open weight models can't comply in the same way, I think they'll be squeezed out of the market. To me that's the trajectory that we're on right now, it's very clear. So I just think that if the open source community doesn't get as politically active, I do think that in the next years there is a great threat that it could be regulated out of existence. And look, I think we're in pretty good shape for the next two years. I know, you know, President Trump may not be universally popular in that room, but he does favor a more lightweight regulatory approach that I think would be beneficial for the open source community. But you know, I think that the political pressure is building I'd say only building the other way right now.
What if you could steelman the argument or help me and us understand better? It seems like even if the US creates a regulatory framework around open models, it just lets China win that race. And so how do people who are advocates for sort of locking it down in the US, what's their and because they're worried about AI safety incidents, what's their answer to the Chinese models?
I mean, they live in a fantasy world where they think they're going to get an international agreement with China. If you listen to kind of the, you know, again, this doomer op that's happening right now, Jacob Coxin, who just left Anthropic, he says that we need an immediate I guess he's calling for an immediate pause, but that that pause needs to be internationally enforced and we need to agree with China. Obviously, they're not going to agree to that. And we know that because so many people who've actually talked to the Chinese AI community, they don't see it the same way that we do. They don't have this like, you know, they don't have this EA lobby that is pushing the other way, creating this doom and gloom. They see AI in a much more pragmatic way. They just see it as something that's going to advance their economy. And so, and their officials already said they're not going to agree to this type of thing. And they don't see the need for it. And more importantly, their population doesn't see the need for it. Like I said, they're 80% AI optimistic. So, they're not going to agree to a pause. I think there's like limited things we might be able to agree with them on. And I'm not opposed to talking with them and having the conversation and just seeing where their heads are at, but no, there is not going to be a deal with China on this. And so yeah, I just think that that is like a huge flaw in the argument that the doomers are making.
And you know, their whole argument around you know is around this idea of centralization is that you know first we need to centralize AI in the United States. There really should just be a couple of frontier companies. We're going to manage the competition between them. You know, competition is only a bad thing in their view. It can only lead to a race to the bottom. It can only lead to these companies sacrificing safety. By the way, that's not true in any other industry because companies know that if they sacrifice safety, they're subject to enormous product liability and they get sued into oblivion. So, you know, safety is one of the things you have to get right when even if you are in a competitive market. But in any event, that's part of their vision is that we got to centralize things in the US and then we're going to centralize things by having the equivalent of an international atomic energy commission for AI. And that's why you hear this nuclear weapons analogy. And the problem with it is just that AI is not a nuclear weapon. It's just not. You know, AI is first and foremost a consumer technology. It's used by individuals. It's used by businesses. Does it have a dual use? Yes. But again, first and foremost, it's consumer, not military. And because it is such an important drive for economic progress, it's not the And because the space is so dynamic, it's not the kind of thing where you're going to get an agreement with China because again, they have a different view of how important this is to their economic development.
So yeah, you might get, you know, some sort of, you know, I think you can get an agreement with China that, you know, neither country should connect their strategic weapon systems, their, you know, their actual nukes to AI. At the end of the Biden administration in the last Biden meeting, there was actually an agreement on that. And I think that's a good thing. Don't get me wrong. I mean, we've all seen that movie before. Nobody wants to connect AI to the nuclear weapons systems, but that's something that we can agree on because neither country wants to do that, right? Even without an international agreement, we're not going to do that. When it comes to something like just, you know, pace, quote, pacing the frontier, which is just the, you know, the continued innovation, advancement of the technology, we're not going to be able to find an agreement, and that agreement wouldn't be enforceable in any event. So yeah, I just think that this whole way of viewing the world, which is that the answer to the risk of AI is just to centralize everything. I think it's wrong and it's not achievable in any case.
Thank you. Well, so let's pivot a little bit on open source AI. Let's assume it proceeds and it thrives. So far what we've seen is that China is winning. All the best open source models are coming from China and there's examples of American companies using them. I you know Harvey is an example of a service that uses is built on Kimi K3 and I work a lot with open source projects in both the AI and Bitcoin world and the past six months has been quite a time with AI security scanning and finding vulnerabilities. And there's you know Kimi K3 has been used by numerous projects to defend themselves against attackers using frontier models. So, I guess the question I have for you is like why isn't America winning and what does America what do we need to do either from a you know is it from a free market standpoint or policy standpoint for America to have excellent open models?
Yeah. I mean, so I mean, I would slightly take issue with the premise there in the sense that I do think that America is ahead of China, you know, at really at every layer of the stack in the AI race. When you know whether it's applications or models or chips, there's really only two areas where China does have an advantage. One is with energy, you know, with net new power generation. They are growing their grid really fast and we're not and that's something we have to address. And then the other like you said is that their leading AI models are open weight our leading AI models are closed and so the market has kind of evolved in kind of it's almost like it's not what you'd expect right because you'd expect it's kind of it's a little bit ironic because you'd expect that you know our system of government you know we like to think of it being more open you know their system of government one party state we think of it being more closed. So it is you know ironic that their best models are all open and our best models are all closed.
I think that just has to do with the business model and how the business model evolved and you know, our companies were first and it does appear that the most profitable business model if you're a leader at the frontier is just to make things close and try and capture all the economic value for yourself. Whereas if you're trying to pursue a fast follower strategy and catch up then open is a great open source is a great strategy because you get all the non-aligned developers to want to be part of your community because they can use it and participate. So I tend to think that's probably what's happening. I do think though that, you know, just the one little thing I would take issue with in your description is I do think that the US has a very vibrant open-source community. It's just and like you said, Harvey is a really good example of that, right? Where Harvey is taking open models and post-training them. They came up with some new models. I think one of the best American open models is Thinky and that was also I think trained off of or still from I think it was like Kimi K2 or 2.5 something like that. So we do have a vibrant open-source community and you look at like the inference companies which are basically creating APIs to open models you know things like open router you know so and they're like growing like crazy so you know companies like fireworks and base 10 and so you know they are huge users of open and so I think the open community is vibrant but you're right that at the layer of the model itself. China is the one producing the best open models right now. It might change in the future. We'll have to see.
So, you mentioned it's kind of ironic that China's using the open playbook because that's not historically how they've gone about things. That said, I think it's natural in business if you're not in the leadership position, you'll turn to open and that probably partially explains why the Chinese models are doing that. So let's dive deeper into there's very significant US companies that aren't in that leadership position but they're well resourced and they're very much in the AI game and that includes Google and SpaceX and Nvidia and Meta among others. I'm curious and obvious, you know, Nvidia certainly been active with M&A and there's certainly activity from each of those entities in this space, but I'm just curious what you view their future is in terms of embracing supporting and investing in open source AI.
