If I was looking at a chart of your overall token consumption, what would I mean? I assume it's upward sloping despite some of these efficiency efforts, but like, how steep is the slope at the moment?
So since January, I think, token consumption has gone up 86 times. Wow. So it's really, really quite incredible. We were not expecting usage to be the way that it has been. And it's been across the board, where it's not just being in the tech and tech-adjacent departments. We have seen people in finance, people in operations, people in the people team using agentic coding workflows. And as a technologist, that's just super exciting to give this new capability, to see this new power to people who haven't been able to use it before.
Hello and welcome to another episode of the Odd Lots podcast. I'm Joe Weisenthal and I'm Tracy Alloway. Tracy, I'm very interested in AI in a... no, no, I'm really, really, Joe, I am. That's a surprise. I'm very interested in the investment context specifically. I mean, the actual implementation of like, how do investors use it? Because I think obviously just sort of substantively, an incredibly important question for reasons that need no explaining. But I also think it raises very interesting sort of puzzles about what the technology is used for. And I remember like when ChatGPT came out and people were like asking it like, what stock should I buy? Or we did that prediction market episode recently and it's like, one thing you definitely can't get much value out of is saying like, which contract should I buy? Or what's inflation going to be? So I can trade this contract. So like, but that doesn't mean that there aren't interesting ways. It just seems like in, you know, the sort of the most crude version of quote, using AI for investing is obviously a dead end, D.O.A. Here's the question I have. You know, we've been through technological revolutions and investing before, notably, you know, we had robo-advisors. That's right. Remember that we had systematic investing once we had high-frequency trading. My big question is how much of the current use of AI is basically an iteration or an improvement? Yeah, some of those kind of, you know, machine learning dynamics, let's say, versus something more substantial or more revolutionary. Is it like a minor evolution or a moderate evolution or something big?
Well, this came up a little bit in our conversation with Aaron from Hudson River Trading. And I'm glad you brought up the quant example because, like, all right. So, like, big data in some sense has been part of quant since the very beginning. Like, how do you establish that, quote, value stocks outperform expensive stocks? Well, you just need like a lot of data and computers and running that math and etc., to establish that fact. But like this idea of like, yes, maybe cheap stocks outperform expensive ones. Momentum stocks outperform stocks with bad momentum. Things that seem to be true, but we don't really know why. And there's like a lot of disagreement as to the source of these things. And that really makes me think about AI, because we can get these outputs from AI models that are obviously, like, remarkable. You can recognize patterns, yes, but they can't really explain how they arrived at that pattern. And I'm very interested in this sort of where this leads us with investing and whether it's like, okay, do we get these like ideas and strategies, etc., that work, but we can't really articulate them in plain English. I'm glad you brought up data as well, because one thing you hear at basically every finance conference nowadays is the importance of data when it comes to running LLMs and making sure that your data is actually processed, it's clean, that you have a lot of it, and that you have hopefully proprietary data. And the people we're going to talk to have a lot of data actually. Totally do. Just one more thing, though. It's a very... I don't know if you caught it last week. We're recording this July 7th. Last week, while you were on vacation, there was a very interesting paper out from Bridgewater talking about their use of proprietary data to fine-tune an open-source version of Qwen, and for a specific purpose of being able to identify what is newsworthy financial information and not. They found that the combination of open-source models plus proprietary data got them better results than the most frontier US models.
Yeah, that is very interesting to me. And like, these are the types of things that I'm very curious about. Where does the secret sauce or the alpha actually come from? Totally. Well, yeah. Especially when everyone is going to have access to like really, really intelligent models. Anyway, enough talk from us. I'm really excited we have the perfect guests to talk about this. Actually understanding the implementation of AI within the investment or asset management context, we are going to be speaking with Gary Collier, Man Group CTO, as well as Tushar Fernando, the firm's head of data and AI. So like I said, really the perfect guests. Gary and Tushar, thank you both so much for coming on Odd Lots.
Great to be here. Thank you. Thanks for having us.
Let's start with Gary, or maybe, you know, go both of you one after the other. Why don't you just give us a very quick description of your roles at Man Group and what specifically you do?
So I can start with that. Perhaps useful to give a bit of context of the firm. I describe Man Group really as a full-spectrum active asset manager. So full-spectrum thing, we cover alternatives, we cover long-only, we cover public markets, we cover private markets, we cover quant, systematic, cover fundamental discretionary investing. And we've got a solutions business that can bring all that together into custom mandates. And what that means in terms of CTO role, I describe that as like full-spectrum too, because for all of the above, we're very opinionated in what every part of the tech stack looks like, right down from choice of servers, networking devices all the way through to the end-user facing pieces of software. And ultimately left to right, we build an awful lot of our own technology right from the custom data feeds all the way through research frameworks, trading systems, and the middle and back office operating platform. So very much full-spectrum role to SRE.
Yeah. So on my side, I look after data and AI. So that's everything from market data to alternative data to all of the context and knowledge that we have that we plug into our AI models. And then from an AI perspective, it's about how do we give the great capabilities that we now have to our quants, to our researchers, and to our fundamental portfolio managers?
Yeah. Tushar, I wanted to ask you about this because I think your title actually used to be head of data and machine learning, and now it's head of data and AI. Like, what exactly, how do you think about the difference between those two terms, machine learning versus AI?
That's a really good question. I think it links to one of the earlier points you had around whether it was that we were using traditional machine learning techniques or whether we were using generative AI. And I think historically, machine learning and AI at Man Group was really about using these more traditional machine learning techniques that were looking at things like linear regressions. They were looking at neural nets to try to predict some feature. Whereas now with the onset of generative AI, it's more than that. It's really about enablement of people to create things as well as using these traditional techniques. So that's why we've changed the title of the role, because it encompasses both the generative AI aspects as well as the more traditional machine learning methods.
Okay, that makes sense. So if I was sitting within a Man Group office today and I take, you know, Gary laid it out perfectly well, you guys have a lot of different roles. So you have discretionary sort of traditional portfolio managers, and then you have more systematic quants and all of that. But if I'm shadowing one of your traders or PMs and I'm watching what they're doing on a screen today versus what they were doing on a screen, let's say 2022 or something like that, what exactly has changed with the use of generative AI? What are you doing differently?
I think if you were to walk across the floor, you would see the different windows, different tools in use now, regardless of role, whether it be trading, quant research, or discretionary investing. And so chatting to a couple of discretionary analysts last week, and they were commenting to me on how everywhere you look across the discretionary floor, everybody has got an AI-focused window on their screen. And so it's affecting, improving, like augmenting pretty much every role we have in the firm, of course, and in different ways depending on what the role is. But there's no role that's unaffected by AI.
What would be a query like? What are they doing? Let's take... let's again, rather than talk about... we're going to have to kneel down. Let's say I'm like a sort of traditional long-only stock selector. Like the sort of classic thing that in our minds we often think of as an asset manager. And it's like, okay, I have now access to some of the world's most advanced models. What am I doing? What am I doing with them?
So if you think about traditionally how that would work, they want to look across a broad set of names, and they want to get access to as much data as possible for those names. So they want to look at earnings reports. They want to look at broker research. They want to look at alternative data. They want to look at podcasts. But there's only so much time in the day. There's a few hours in the day, and there's 50 names. How are they going to cover them all? How do they get to the really important insight? And what AI has allowed us to do is allowed us to access all of those different types of data, lots of different modalities, podcasts, alternative data, things like broker research reports, and synthesize them into what's actually meaningful. So the PM can asynchronously get that insight. They could go away for a coffee. They can come in overnight. And then the agent has actually distilled a change that's happened on the internet and given them the insight that's meaningful to their portfolio, to their investment thesis. So just, I suppose, an example might be that recently we had a PM that was covering the AI trade, and really one of the important things is about where's the bottleneck? Where's the bottleneck here? And yeah, it's the bottom, is it? Yeah. It's hard to know exactly where it is. And there was a podcast from one of the heads of engineering from a large hyperscaler. And he was saying that it was increasingly important to have more and more GPUs to train models, but data centers were becoming more scarce, and it was becoming increasingly difficult to find data centers that were actually big enough to train these models. So what you could see, there was a couple of things. Like one, that was GPU scarcity. And the other thing is that it's likely that we're going to need better networking between these large data centers in the future. And that was something that came on a podcast from the head of engineering. This isn't someone who usually interacts with... they don't go to the investor calls. They're not somebody they often have access to. But through AI, the PM was able to have an AI agent transcribe that podcast and that synthesized that data so that they could get better insight into that investment idea.
Yeah, I feel like podcasts are an important source of alpha. Everyone should be listening to podcasts. You know, the non-joke version of that, which is one of the things people say, we had Alex Damour on the podcast and he's like, what's going to be scarce after AGI? Yeah. People who are able to get good guests on their podcast, people who are able to get those people, you know, okay. So we're clearly talking our own book. Yes. Okay. Setting that aside for a second, it sounds like what most of you're doing, and when you describe that process, is augmenting research or allowing your investors or managers to be more efficient in their research process. There's a lot of talk nowadays about agentic workflows and the actual idea that instead of having humans drive every step of a particular trade or project, you have a system that can, you know, do the research, it can generate ideas, it can test the hypotheses, and then it can even execute on them. Is that something that you're sort of working towards, or is it still too far away for you guys to even be contemplating?
No, no, that's not too far away at all. In fact, that's something we've been like working on for quite some time now. And if we shift focus to the quant, the systematic side of the business, what we've been doing. Now, if you take a step back and think, well, technology plus quant techniques, that gives you the ability to build systematic strategies. So what do we get if we add AI into the mix? Well, we have the ability to think about systematizing the way that we build systematic strategies to begin with, and in effect, giving a big force multiplier, a big leverage multiplier to our quants, our researchers. So what we've been doing there for well over a year now, actually a year and a half, is building a system that can actually take all of those different parts of the, quote, research process. So the idea formation part, looking at academic papers, looking at carefully labeled data sets, reasoning about the contents of those papers, the contents of those data sets, are there economic hypotheses that could be real or at least could be worth testing out, and then having other agents build the code to encapsulate those ideas, get the right market data, run the right backtests, etc., and then further agent search evaluate the output of a prior agent. So this is really one of... this is early ideas that we thought we would give potentially great bang for our buck. We were working on some time, and just to make it look real, to demonstrate this is not just something that's happening in the lab, but doesn't have any real practical consequences. There've been a bunch, I think 15, 20 models at the last count. That's it. Gone all the way through. They started off as models that were ideating by... I went all the way through. They signal construction. The validation process were reviewed and validated by a human investment committee and things like fit and proper to trade our client's assets. Well, so this is very real. It's not make-believe at this point.
Since you mentioned sort of hoovering up information that might appear on a podcast where someone identifies a particular bottleneck in their business, we are on a podcast. What is your bottleneck? What would you like to have more of right now in order to improve your process? Is it compute? Is it data? If you could snap your fingers and have more of something, what would it be?
Well, you can always use more compute and you can always use more data. But I think what the bigger problem is here, the world of opportunity that's afforded by the AI capabilities that we're seeing and, of course, being delivered all the time, the envelope is expanding like so quickly. That's, keeping up with it from a human perspective can be like quite hard. There's no shortage of good ideas. Filtering ideas is as important. But of course, we're used to making lots of change. Man Group. The sheer level of organizational change that we need to get involved in to put all of this stuff into effect. I think that's the bottleneck, making sure, because we're a regulated business, we've got a fiduciary duty. We need to make sure that the things that we're doing, the things we're deploying are done with a minimum amount of risk. And there's a lot of work in this field that really, I think the industry doesn't have answers to yet. I mean, evaluations, testing the results of AI processes, a good proper, that's a rapidly expanding space. Thinking about how we run agents across the business in a safe and controlled fashion. Yeah, we all, I'm sure, have seen and smiled at the... well, I deleted my inbox or I deleted all my photos and I can't get them back. We can't afford to have that type of thing happen at an enterprise level. So it's making sure that we can move quickly, but safely, I think is the bottleneck, if you like.
Yeah, that makes sense. Let's get into a couple specifics, and I want to get back to the sort of how you move forward safely. But both of you have now mentioned filtering, and that actually is precisely what I mentioned in the intro. There was a paper from Bridgewater about fine-tuning a version of Qwen on their own data precisely for better filtering. So that the PMs could just be, you know, have more efficient use of their time to see what signal. What do you... how do you build that? Is this an off-the-shelf thing? Like, what is the tech, the model, etc., that you have to... like this ingestion pipeline. Like, what does it consist of?
Yeah, I can take that one. So I think for us, it really depends on the type of data. There are broadly three types of data that we look at. The first one is market data, that's often very structured data. Things like order books we ingest every tick from most exchanges, almost a terabyte of data just from ticks per day. And then we have alternative data and unstructured data that is much more messy, it's much more malformed, and that it's really about how we structure the data, how we tag it, how we connect it with each other. What's the knowledge layer on top of it? How do you think about the connectivity between tickers and sectors and companies? How do you actually interrogate that data in a way that is uniform? What's the common language between all of those data sets? And then the last piece is really something that's quite new. It's institutional knowledge. It's all context. And this is becoming a new layer in our data architecture. How do we tell AI about our processes? How do we make it speak our language? How do we tell it the right way to run a backtest? So those are really the three areas that we focus on.
Just pushing on this point further, like all of that makes a ton of sense to me. And especially, you know, the second two are like big problems. And we know that generative AI in particular solves a lot of the unstructured data problem. You can actually get a lot of signal from, you know, a bunch of text dumped in a file. But what are you building? Like, how does it work? And like, do you have to... can off-the-shelf frontier models... are they the best for the job? Do you do your own in-house fine-tuning of open-source models like right today? What is the best tech stack for that part of the process?
So we have historically looked at fine-tuning for a couple of cases, but where we're seeing the most bang for our buck at the moment is proper tagging and structuring of the data. So pre-processing. What we found is that if you take a data set like credit card data, for example, I can look at that, I can see the columns, I can see the rows, but it doesn't really understand the nuances of it. It doesn't really understand what it is. So what we're having to do is invest in trying to add extra color, extra metadata to that information. We want to have descriptors to tell it the nuances. We will say things like, when you look at this data set, each row really means that a person has gone into a shop and bought something. And you tell AI that in plain English, you give it those descriptors. Yeah, and you do that for all of your data sets. And then the second piece is how do you connect lots of data sets together. They use very different language. They use very different semantics. So investing in a shared language, a shared semantic layer, or a shared way of doing things is something that we've had to do so that we tag all of the columns with this unified language so the AI can connect different data sets together, because that's really an important thing in idea generation. That knows the AI field in one data set is linked to a field in another data set, it can quickly navigate between tickers and the sectors for macro insight.
What do you think is most important? Having the latest frontier model or a beautiful set of structured, tagged and labeled data?
Yeah, I'd say it depends on the task that you're trying to do. If you're looking at a coding task, you really want the latest frontier models. But for a quant research task, I think looking for the underlying data is what fuels alpha research. Trying to just use a frontier model will get you nowhere.
So this is something that comes up in a lot of our AI deployment questions. And I don't know, maybe both of you can take this, but you have all these different teams, and I assume everybody wants really, including people who don't know much about AI, intuitively think, oh, I want the strongest version of the model. I want Opus 4.8, I want Fable, whatever. And I assume like, okay, yes, for like deep computational tasks or solving engineering problems, coding problems, yes, probably those are the best. But there are probably a lot of people who do not need anything like that at all. How do you think about the question of internal provision of token consumption and not wasting money by having people use the most advanced models, but also giving people enough green space to explore and figure out what is the maximal potential value that they can get from... I guess the economics question is an interesting one. It's an important one.
What we've done there is, well, towards the end of last year, when we knew growth was likely to accelerate rapidly, we modeled what we thought the company would start to look like in terms of different classifications of users with different use cases, came up with certain budgets. And then what we've done this year is federated data out to all of the business units within the firm. I'm a strong believer in pushing decision-making down to those at the lowest possible level, that it makes sense to allow people to be agile and use the economics at work for them and their departments. And of course, budgets are fungible. If departments want to move money from some other spend and say, right, I think we should buy more tokens, they are free to do that. And the platform that we've built, the AI platform, has a very rich... not 100% complete, but a very, very rich set of models that can be chosen, including all of the main frontier models and a number of the different open-source, open-weight models.
Do you build a model that routes queries to the optimal sort of like cost-efficient model? I know there's a lot of interest in this. The sort of classifiers and the big AI labs have them themselves, but the classifiers that route queries, is that something that you have in-house or how do you solve that problem?
