Change is the only constant here, very rapid change, not just change, right? Is the person who joins this company coming in with a mindset where they are willing to adapt and change very quickly and learn very quickly. Being a generalist is not going to help. Pick something that you really love and go really deep because you will be able to bring emotion to it, you'll be able to bring personality to it, which AI cannot do. And leave all the general things to AI.
Welcome to Super Companies, my quest to understand how the smartest and most valuable companies will be built in the age of AI. I'm your host, Greg Shove, seven-time founder and CEO of Section AI. This episode we're speaking with Sid Sheth, co-founder and CEO of d-Matrix. Sid and his company d-Matrix are building inference chips to make AI cheaper and faster because the world's going to need a lot of cheap and fast AI, which is why he's raised $500 million, has a $2 billion valuation, and only 250 employees. Sid is the CEO of a potential super company. He's going to chat about his philosophy around managing change, the moment when he realized that his workforce would truly be an integration of agents and humans. I think you'll enjoy this conversation. Sid, welcome.
Thank you, Greg. Thanks for having me.
Yeah, and you're in the Bay Area, obviously you're in the South Bay, and this is where the company's based. And are you guys in person?
Yeah, we're in person. We don't have a mandated policy that you have to be here every day, but we do encourage everyone to be in the office three days a week, and it's been quite successful, actually. So many exciting things happening. Actually, people want to come into work.
And how often are you in? Are you one of these leaders that actually does it or goes to your house in Napa and tells everybody else to go in the office?
Call me old school, but I am in the office every day before 9:00, and then I leave late.
And do you have co-founders? Tell about that.
Me and my co-founder Sudeep, the CTO and co-founder of the company, he focused more on the technology, obviously. I focused more on the business and financing and fundraisers and whatnot. We have known each other for a very long time. The previous business that I built was a company called Inphi, where I built them. They are a data center business. Sudeep was my partner in crime there, where he was a CTO for that business. That business eventually grew to 3 billion in revenue from zero. We have been married before, and this is not our first marriage.
I'm going to ask you a question about super companies. And if you agree, great. We're going to have a great conversation. If you don't, this might be a short episode. Before I ask my question about super companies, tell us what d-Matrix does, but keep it short and dumb it down for me.
Yeah. So, I think everyone has heard about training. Everyone has heard about inference.
The answers you get from AI, basically, right?
That's right. Every time you query ChatGPT and it responds back, that's inference, right? We are building a solution that really speeds up how quickly AI can respond to humans.
What was the insight you had in 2019? That seems pretty early to me.
I've been in the data center business for, God, 20-plus years now, when people were not even building data centers, right? Because we went into the world of cloud computing, and then AI came along. We had a ringside seat to what was happening with AI, because the AI transition had just started in 2015, 2016. And the chips that we were building at Inphi were being used to connect AI computing clusters.
And so, you were selling those chips to OpenAI and those guys?
OpenAI was not around still. It was Google, Amazon, Microsoft, all the hyperscalers deploying these chips to connect GPUs. And these clusters were getting bigger and bigger. They wanted more bandwidth. That is what led to us probing and talking to customers and trying to understand, what is it that they're really trying to solve for? They educated us. We learned a lot about the fact that once you train these models, they're deploying it in their fleets. They're using it to make their search better. They're using it to make tools better, right? But it was really restricted to internal use, right? For the most part. It was not really available to the people on the outside.
And the education that they gave us was there will come a time when these AI models will become substantially capable, where they are very applicable to not just the internal tools and properties that these hyperscalers have, but will become really applicable to everyone on the outside. What does that world look like? Because now you suddenly need gobs and gobs of inference because you're deploying lots and lots of AI. In that world, what are the constraints? The constraints are going to be compute, it's going to be energy, and efficiency suddenly begins to play a very important role because when you're deploying inference on a global scale, we have only limited amounts of energy on the planet. So, how do you effectively run so much compute with the amount of energy you have? And more importantly, when you take this out to the broader world and you make it available to consumers, they care a lot about interacting. Speed becomes very important. These hyperscalers might have more patience dealing with AI, but people on the outside don't have that level of patience, right? So, it became very clear to us that we need to build a solution that is dedicated for inference. How do you make it fast? And how do you make it efficient from an energy perspective, assuming that this is going to be deployed on a global scale?
All these analogies we've been using the last two or three years, inference will be like electricity, AI will be everywhere, we'll just plug into it. Is that your perspective? You've been on the inside and early. What do you see in the next two or three years in terms of how this unfolds? That we're all going to have near unlimited access to great inference, and that's the way we're going to live and work. Is that your vision of the future?
That's absolutely the vision. I know there's a lot of fear about what AI can do, but really at the end of the day, it's a tool that augments humans. The ones that will embrace it to make themselves better will be wildly successful, and it's getting to a point, I think AI is now getting to a point where, especially after what happened end of last year with Claude Code and OpenClaw. Those were two watershed moments for inference, right? Up to that point, I think a lot of the talk was about training models and capabilities unleashed from training bigger and bigger models. Then we had the DeepSeek moment early part of last year where we quickly realized it's not just about training big models, you can actually take big models and distill them down to small models and make them more efficient. And then you had the Claude Code moment and the agentic OpenClaw moment. For the first time, you could actually have a layperson use these models embedded into their workflows and use it for daily productivity enhancement. It fundamentally became easy to use, and I think we are only three or four months into this. I just fast forward 3 to 4 years. This is what the world is going to look very different. I think AI is going to be everywhere. I think cost is really going to be the main issue. How do we make this really affordable to everyone? And cost is coming down very rapidly.
Is cost going to keep coming down? What's your sense of what we should expect as consumers or if we're running our own company?
Cost is coming down. If you look at a comparable unit of intelligence from a year ago, the cost is an order of magnitude lower today. The amount of intelligence you're getting per dollar is actually staying flat because while the cost is coming down, the intelligence is going up. You can't infinitely keep getting smarter, right? The intelligence curve will flatten out, but the cost curve will keep going down. And then you will start seeing dramatic declines in intelligence per dollar.
Right? And the proliferation will keep increasing.
Is there enough margin for these guys though? I guess just focusing for now on Anthropic and OpenAI as proxies really for the business model of that part of the business. Do you have confidence they'll find enough margin?
The volume economics are just kicking in. Today is not the time to play the margin game, right? It is really capability and adoption curves that have to be pushed. So, I think about a few years out when the volume economics kick in, 100% they'll be making money. They'll be wildly profitable.
All right. And buying lots of your chips.
Oh, absolutely. That's a must.
Here's my super company question. I believe we're entering the era of super companies. I think super companies are built and managed differently than previous generations of high-performing companies. So, previous generations today would be of course Google and Amazon and Netflix and Databricks. I believe we're entering a new era of wealth creation, of value creation. I think every category in industry will have super companies, meaning their products will be likely informed and leverage AI, but more importantly, they're going to operate internally differently as you suggested with AI. And that one of the ways we'll be able to spot these companies is they're going to be very efficient. They're going to have very high revenue per employee numbers as one metric. Do you believe we're entering the era of super companies? That's my first question.
Absolutely, 100%. We've been saying this for a long time.
Good, because that means we can keep talking. So, what I want to talk about, Sid, is how you're building the company on the inside. Your company's still pretty small in terms of employees, but mighty, and that's what I think super companies are. I would imagine all the technologists in your firm are augmented to the max with AI, I would imagine. So, how's that going, and do you feel like your business functions are equally as augmented or jacked on AI?
This is a very important question. I think to me, it is really about the ethos you build, right? We are living in a time where culture becomes really important. What kind of culture are you building inside the company, right? As a startup, I think you obviously have a unique advantage because you can start clean slate. You're building an ethos, and I think one of the things that we have focused on, and by the way, this transcends all functions in the company, right? This is engineering, business, everything. Is are you willing to embrace and learn quickly, right? So, change is the only constant here. We are living in a world of change and very rapid change, not just change, right? Is the person who joins this company coming in with a mindset where they are willing to adapt and change very quickly and learn very quickly. So, embrace a very growth mindset.
I'm not disagreeing. It does seem like you need growth mindset to the max, though, right? This ethos you have, was it always that way, meaning from 2019, or did you refresh it?
First of all, I am that kind of guy, right? And my co-founder is that kind of guy. So, I think both of us have that ethos to begin with, and then we attract people around us who have the same ethos, right? It's been there from day one. Maybe the AI tools were not there. We didn't have AI to use as a tool inside the company back then. Competition was moving very rapidly around us. Architectures were changing very rapidly around us. And the pace at which all those things were changing only accelerated post ChatGPT, right? So, how do you build a team that can respond to these changes in real time, make pivots very rapidly, focus on what is real, and focus on what will really make impact, and keep the product competitive? All these things are a culture, and then of course, what has happened in the last few months is with the arrival of AI we are using it as a tool to accelerate how quickly we can do all of these things. Right. So it was not like we responded to something that was happening on the outside in 2023 and made the culture around that.
At this 2023 time frame, were you one of those CEOs saying I want everybody using ChatGPT or Claude or whatever? There was a lot of both hype and then CEO kind of edicts. Did you need to do that or what was your mindset?
I didn't need to mandate it because people just adopted it. It had limited uses for a company like ours, but I think with the arrival of Claude Code that completely changed. I mean by February we had rolled it out to pretty much everyone in the organization. So we ran a pilot in Jan, we rolled it out in Feb and now I have appointed my co-founder AI Czar.
Do you expect business people to be building agents or working inside of CoWork or is your expectation that we all should have this builder mindset?
I leave it to each user. I'm not mandating how they use it as long as they use it. And I think what I'm more interested in is not how but what. What are they doing with this? Because every person has a different use case. Now that we have a team that is overlooking all the different use cases, we actually incentivize all the users to send in the different ways they're using the tool. Not only that, we have a kind of a central repository in the company where people can drop in all the different use cases.
But is your expectation now that the company runs on AI internally?
What about the business functions? What's your expectation of a marketer or a sales person or finance person? Should they be AI fluent by now?
Absolutely. Financial analysis, financial reports, summaries, AI is just phenomenal at that kind of stuff.
You're growing, so my guess is there was no concerns about job loss or anxiety. I think larger orgs suffer from this, right?
If there was anxiety I didn't see it. I would be remiss if I said that there was none. As a startup because the mission is overpowering, I think this becomes less of a concern.
My sense of this is in fact bigger companies are slower and startups are accelerating faster. The divide between a super company and incumbent, I actually think it's widening. You know, because I think startups are so able to get these technologies and this augmentation embedded much faster. To your point, Claude Code was last November, right? I think this really is going to be gnarly for incumbents. Are you seeing that? Where do you see the maximum risk? Which segments of the economy or which type of companies are less likely to transition into or miss out on the opportunity to be a super company?
Incumbents have huge advantages, as you know. Brands and distribution and customers and capital and so on. They're not going to be easy to unseat, but it wouldn't surprise me if this kept accelerating.
It's really an opportunity to me. I think anyone who does embrace this has a way to really turbocharge your business, right? The message to the company was, 'Look guys, this is very important to me, and it's very important to the founders. We're not taking this lightly.' And obviously, founders have that special place where they can do things that most other folks would not be able to do.
