One of the advantages that you have as Meta is that everybody's using it for consumer use cases, you're able to feed that back in. Explain how you're training on your own employees. This has obviously been controversial recently with the model capability initiative. What are you trying to learn from your employees? As it's been reported in the press, you're tracking what they type in, learning from their intelligence. I believe it was Zuckerberg said our employees are more intelligent than the average person. We want to see how they use machines so that we can make our AI better. Explain that.
Yeah, although that program is now paused. So
Program is paused? Just for now?
Because of the data issues. Well, we're trying to make sure we have a full accounting for the program.
Let me explain the theory of it. The theory is actually very straightforward. You've got a bunch of employees that are already doing this work every day.
And for no incremental effort on their part, you can potentially improve your models.
And that benefits not only your employees when they go back and try to do their jobs, but also consumers, people who want to use AI. Now, the scope of this research and it really was research. It's worth noting. This is the research that we didn't expect to yield for like 18 months, at the earliest. It would be the earliest that we would actually have a look at what this would look like inside of a model. It needs very long-running data. It was literally things as simple as how to use a computer. A thing that AIs are surprisingly bad at for being themselves, computers. How do you navigate UIs? Like what do you click next? And a lot of our workflows are in these digital silos. They're going to be there for a long time. A year and a half from now, it's still going to be the case that these flows that we're going through every day as we go about our work. So, the most basic premise was like, 'Hey, can we teach AI to use computers?' It's a data set that doesn't really exist
At scale. And you can do acute versions of this. You can certainly hire people through Scale AI or other places to like, 'Hey, give me a quick walk-through.' But we wanted to see, 'Hey, over the course of months, how do people make progress on work literally using a computer.' And so that was the premise of this research.
What were you recording and tracking of them?
Yeah, it was like literally your mouse movements, like what are you clicking? Like where are you, what window are you activating?
And trying to create enough of a data stream here.
And was it all employees? Like were you tracking yourself?
In the US. There was a few employees excluded. So a good example is if you look at someone like me where in a position that I'm in, we're under any number of legal holds.
Where it's like, 'Hey, any information that's created on my behalf is automatically stored and preserved by law.' That makes me a relatively poor target for this work. So there's a bunch of jobs, people who regularly worked on sensitive data. That was like, 'Hey, you're excluded.' So we excluded bunch of teams, we excluded individuals. There's a whole process by which people can opt out of this stuff.
Now, what would make for an amazing data set? How do executives at companies dealing with legal holds handle computers? That's a proprietary data set that you can imagine having got you.
Yeah, taking a step back, I think we're in a very interesting place where as you saw the trial, the OpenAI trial with Greg Brockman's writing a note to himself as a diary and that's being exposed. And they've now I think there is at least one legal precedent that shows, 'Hey, asking questions to an LLM is not you can't have that be protected even if you're asking a legal question.'
I do think there's an interesting question that we need to raise at some point relatively soon on privilege and what should be privileged and not privileged in a world where you can credibly get a consumer, somebody who couldn't even afford a lawyer necessarily, could get credible legal advice. Set aside, nobody cares about corporations. They got enough money to hire all the lawyers. Somebody who can't afford a lawyer but could actually get expert legal advice, the premise of our system is that people should have legal representation. Like what does it look like? So there's an interesting question but I'll leave that for the courts.
That for the courts and the legislators.
All right, let's go back. So, you and other people who work on sensitive information are excluded from this program, but the majority of Meta employees of American
Yeah, yeah.
Right. Presumably you think like Europe would never let you do this, right?
