Palantir’s Politics, Elon’s Approach to DOGE, and AI’s Effects on Writing w/ Byrne Hobart

Yeah, I've been doing a little bit of that recently.

Yeah, so I was thinking about this a lot because I will sometimes encounter this idea that up until about 2023, you could assume that anything you wrote was written by a real human being, and that kind of meant something, and now you can't really assume that. And I think that is technically true — it is true that LLMs can put one word after another in a coherent way, and they used to not be able to do that. But the kind of writing that they produce is still, one, it's still kind of detectable. You still get the vibe — you'll still notice this in the occasional email or Twitter reply or whatever. And I don't think it's a huge deal. But also, a lot of the writing — being able to produce 95th percentile writing is an incredible technical achievement, but the 95th percentile is not nearly adequate for competing in the market for quality writing. And to the extent that it actually becomes hard to distinguish, that's just a combination of one, technology advancing, and two, not everyone has great taste. And in a lot of domains I don't have great taste either, and so I shouldn't be really surprised if AI is actually writing some of the stuff that I read. Like if I'm reading the wiki for a complicated strategy game that I don't actually have time to play but I want to play, I wouldn't really be able to tell if this part of the wiki is written by a human being or by an LLM that has just hoovered up a bunch of information that other people have posted about the game and is reproducing it. But I don't really care, and I'm not really reading that in order to understand what any one person thinks or in order to have some sort of parasocial relationship with the writer. It is more just that I'm looking for information, and a convenient way to display information for people is to format it in natural language. Now we have a cheaper tool to do that.

I actually think in the piece I wrote about this that one of the bigger social adjustments — a larger social adjustment we've had to make — was actually the adjustment to the existence of blogging. Because it used to be that if you wanted to write something and have a lot of people read it, you wanted to write about current events, you actually had to follow some kind of path where you start out in a local newspaper and you're covering City Council meetings, and it's incredibly boring, but you do have some chance to just show that you're reasonably competent and that you can write down the things that happened and you can highlight the important things, and you have an editor and so on. And then there was this path where people would go from that to reporting on higher profile stories — sometimes it's because they get promoted, sometimes it's because they get lucky. In other cases people would go more of the commentator track. But it does seem like it was hard for a long time for someone to just directly break into the business of having opinions on things and writing about them.

Meanwhile, this whole ecosystem that has sort of a status hierarchy, has a standard path, etc., it also had a lot — it was also kind of defined by just the technological limitations on publishing. The news that was read is something that happened once a day because the newspaper comes out once a day, and then for commentary maybe it's once a week or bi-weekly or monthly because you're doing that in a high-status magazine. But there was this established cadence for how fast the news happens, and that was already accelerating because TV news can be pretty real-time, radio news of course can be real-time. But then what blogging did was it meant that writing — which is just the way that important people consume information and will be for the foreseeable future — like, smart people are really not going to switch to pure TV, pure audio. They will consume those things but a lot less than average. They are getting most of their ideas from the written word, and to the extent that people are audio and video consumers who are important people making important decisions, it is usually not consuming something that is meant for a mass audience. It's usually something where the production values are not great and there's not a lot of flashiness in design — it's mostly just a conversation.

So something like — I could buy the idea that smart, well-informed people were watching William F. Buckley on Firing Line and that they were TV watchers, but I would also buy the idea that the median, the typical member of the Firing Line audience, that was the TV that they watched — it was just William F. Buckley and nobody else. So I think that's a general bet to make. But with blogging, people who were just generally talented, generally informed, or just really good at zippy terms of phrase — they could actually get famous purely online, being totally outside of the normal journalistic hierarchy. And blogging is a lot more real-time than other media, so they were just inside the loop — they were reacting to stories faster than the mainstream media outlets that were producing those stories could actually respond.

When I was going back and reading some of the history on this, I was struck by the fact that I always remembered that there was this phrase where someone in the traditional media had talked about 'pajama-clad bloggers,' and it was of course a derogatory term. I was wondering how that came about, and the actual reason it came about was that the first big story that was broken by the blogosphere was this minor scandal — I mean it was a big scandal at the time, but in retrospect a very minor scandal — which is that CBS did this report on George W. Bush's National Guard service where they said that he had skipped out on parts of it. There's enough circumstantial evidence that you basically have to believe that George W. Bush was really, really patriotic for a slightly CED-up frat boy from a very wealthy family who didn't want to go to war in Vietnam — for people in that category, he had to be extremely patriotic to actually show up to everything, but also disorganized enough that sometimes the paperwork did not reflect that, etc. CBS had this report where the blockbuster anchoring it was a memo — the memo was talking about covering for George W. Bush and admitting that he was not actually fulfilling his duty, etc. And someone — I think it was the Little Green Footballs blog — pointed out that this memo was actually not written on a typewriter in the 1970s; it was clearly composed in Microsoft Word and then artificially aged to look like an older memo. So that was actually a pretty good story — obviously a very, very in-the-media story. It is not really a story about a thing that happened in the outside world; it's a story about how things were covered. It makes a big media organization look embarrassed, makes a smaller media organization look great.

But that was actually what prompted the original quote about pajama-clad bloggers. The bloggers were not really an issue until they started actually directly competing with the media — not just breaking stories, but breaking stories about bad things that larger media companies have done. But what actually happened was media companies all adapted. A blog is now just a standard thing that any large media outlet will have — cable stations have them, local TV stations have them, newspapers have them, magazines have them. It just got totally incorporated into the existing media setup. But it was a really tough adjustment, and I'm sure if I had been a magazine editor in 2003, I would have thought that one, blogging is really stupid; two, these bloggers are just not as confident as the people that I've hired — you read a random blog, you compare it to a random Atlantic piece, there's a really clear skill gap; and three, even if they achieve parity in terms of the product, at the time monetizing your content — which is absolutely not how anyone in that industry at the time would have said it — but you wanted to do that on the printed page. You wanted really glossy, high-resolution ads. You knew that these ads were being delivered to a particular audience — you had the zip code and the address, so you actually knew roughly how wealthy they were. There were lots of reasons to prefer that versus the pennies that you get from display ads on a website. So there were lots of reasons for large legacy media companies to resist this, and then eventually they incorporated it.

And now we just kind of expect that news happens all the time, and that to the extent that you are reading printed news, it is either that you're very, very old, or you kind of like this sort of larpy ritual of unfolding the giant Wall Street Journal at the breakfast table — although actually the Wall Street Journal is now smaller than it used to be so you don't even get to do that — or you are reading something that is meant to come out in a lower cadence and it is meant to be not the first word on what happened but the last word on something that happened. So for something like a cover piece in The Atlantic, there will be cases where it needs to be published in March and it's pointless to publish it in April, but there are also cases — and the better the article is, the more likely it is that no, the month in which you publish it is the month in which that's the biggest story in that part of the world. And so you just wait until you get it right, and when you publish it, it is necessarily timely because you made it timely.

So there's been an adjustment, and I think that adjustment has actually helped a lot of media companies. It's helped them really tighten up what they're doing because they know at this point that they are not going to compete with Twitter to be the first place where you read the first little blip of information about some new breaking event. It's very, very hard to beat that. In the cases where you can beat that, you pretty much have to invest a lot in having the writers who have the access to get the story, etc. And I'm sure there are cases where there's something time-constrained where you know that if you publish that story on NewYorkTimes.com in 15 minutes it's a Times exclusive — if you wait 30 minutes, triple-check some things instead of double-checking them, etc., it's somebody else's exclusive instead. But in a lot of cases they've kind of recognized that there are these different cadences of the media business, and that it is just blurred across this timeline instead of occurring in these discrete chunks where if it happened in time for the story to get finished, in time for it to get typeset, in time for it to get printed for that morning edition, then it's part of that day's news, and otherwise it's the next day's news.

So I think that was a big adjustment. I think the other big adjustment that people kind of sleep on is spell check and grammar check — that these are all ways that people can get the quality of their writing from whatever percentile it is naturally to adequate, good enough. Good enough for intra-corporate communications, good enough for emailing someone who is part of just a different social stratum than you. And LLMs have both — they've brought up the bottom end of that distribution because they are better than a word-by-word spell check tool at figuring out what you meant if you are in the tenth percentile of writing ability and so it's really hard to know what you meant, but an LLM can usually puzzle it out. One of the first kind of heartwarming applications I saw for — I think it was GPT-3.5 — was that someone built a tool where his friend who had a yard work business but also was just not academically inclined at all, his friend could take emails that were just written in maybe SMS jargon and convert them into a normal email that you would send to the upper-middle-class customer of your yard work service. So tools like that I think are just a great force for egalitarianism. But they're also an annoying force in the sense that you can no longer instantly prejudge the quality of someone's thinking by scanning the first paragraph for typos. So you do actually have to read more stuff to judge people. It does mean that people who receive a lot of inbound emails from people of just a variety of writing and thinking skills can no longer assume that the correlation between those is strong enough that you can just treat typos as evidence that you should ignore something.

So I think that is an adjustment where we'll continue to make it, but it's also an adjustment we have been able to make socially. The fact that we mostly send the written word, we mostly type the written word and then send it digitally rather than writing it out, means that just the way someone writes is less of a class signal than it used to be. But I think we'll probably handle it just fine. And it's already the case that there's just a huge surplus of written content and that most of the content you could hypothetically see you will never see, because the people who are designing the algorithm for your newsfeed and the people who are editing whatever publication RSS feed you subscribe to, etc., these people already know that most of their job is to exclude things. Most of their job is to reject things that are pointless, that they shouldn't write about, that their audience doesn't care about, etc. So a lot of the work is already getting done, and if it gets a little bit harder, I think that is still survivable. And I think ultimately, if LLMs are actually producing writing that is on par with the writing that human beings were doing in some particular domain, then this is just another case where there was a job you used to have to have people do it, now you can automate it, now those people are hopefully doing something else. But either way, we're very unlikely to go back. So that is an adjustment — maybe for people in the media business, but in particular people who are in the media business and whose views are very predictable based on things they've already written or what's in the corpus related to the publication that they wrote about.

So like, it might be — and this is something I've actually done, I have tested — one of my newish benchmarks for LLMs is just if I'm going to write about some story, I will give the LLM the story and ask it what I would write about it. And it will sometimes get the central idea right, sometimes get it wrong, often doesn't have the same examples I would use. It's worse usually — there hasn't been a case where I looked at the results of that and I was like, okay, I actually need to find another job. But it has gotten better over time, and so I try to just mentally be prepared for the fact that either those improvements level off, or at some point an LLM can do some of the things that I do and I need to adjust the way I think about what it is that I do.

You've already spoken to some of this and also alluded to it, but I want to point readers to it and also give you any space to expand. You wrote three sort of segments related to news: one is sort of real news remaining the least lucrative part of the news business; two, the different kinds of subscription businesses — one that tells you information nobody else knows and one that gives you the opportunity to fit in or believe the same thing as your colleagues or peers; and then three, one on the Washington Post and how it's sort of changed, catering to a certain customer set with their changes.

Yeah, so I could take those backwards, I don't remember which order you said them in. But yeah, the Washington Post one — it's possible that Bezos took over the Washington Post a decade ago as part of this long-running conspiracy to make it a large libertarian newspaper, but I think what actually happened was he was looking at the existing media environment. He realized that you've got the New York Times — they are progressive, they will always be progressive. You've got the Wall Street Journal — they are a conservative paper, but they're also a conservative paper that is trying to be cooperative with the leading conservative parties in the places where they operate, and so that means it's a paper that gets trumpier. And so if there is some audience that is the combination of progressives who are in favor of growth, who think that if we want to redistribute a lot more to the poor and solve a lot of their problems, that's just a whole lot easier if there's more to redistribute — so let's go ahead and build more things and get cheaper energy so that we can raise the top marginal tax rate and provide more healthcare to more people, etc. And I think that's a very internally consistent view. We'll see if it can actually survive the Democratic party apparatus, but early signs are actually kind of promising.

And then on the Republican side, you have a lot of Republicans who are just pretty aghast at the idea of a president raising tariffs. They view tariffs as a tax, which they don't like, and as an unusually inefficient tax, which they like even less. So there is this audience that is being underserved by large media outlets, and then there are plenty of smaller media outlets that do actually go after this audience. So if you want the more political version of that, you read Reason magazine. And if you want a more centrist libertarian and pretty establishment-friendly take, you would read The Economist. But there's also room for a daily paper that is a recognizable brand name that is going to go after that audience. And it's not a huge audience, it's not a huge constituency. It's very hard to get truly libertarian parties elected, except in places that are just utter basket cases. So Argentina is a very special case, but Argentina is almost this demonstration that you may want a more libertarian head of state and head of government, but you probably don't want to live in the kind of place where that is actually the most popular option. You should probably be really, really happy that center-left and center-right are the main things on the menu, and that major parties try to sound a little bit less extreme — that's what they compete on, being more centrist. It may be a bad thing prospectively, but it's a good place to be retrospectively.

So that's the theory of the Washington Post. And then yeah, we have the general theory of hard news — why it's a bad business, which I still stand by. It is just not a good business, but it's worth thinking about why that kind of news is typically a loss leader for both the print-turned-text publications and for video publications. And there are some really obvious causes for that. So if you want to report on celebrity news, you just have your journalist in places where there are lots of celebrities — they will see things, they will overhear things, etc. And if you want to report on international news, you need someone in every place where crazy things happen, you need someone who is motivated to go to places where these kinds of things happen, you need someone who is tolerant of the risk — like if they really want to report on a coup, they also are going to be the person who's on the ground snapping photos and writing up their notes during an actual coup. And so the odds of getting shot or otherwise experiencing some very unpleasant things are a lot higher for those people. So there's this high fixed cost to doing that.

But I think the deeper reason — so I wrote about this a little bit — is that you can think of actual news, it falls into two categories economically. You've got news where it's valuable to you because you know something other people don't know, you can act on it before they do, and you will either make money or achieve influence or otherwise reach some of your goals. And the natural way for this news to be priced is some combination of there's a very high subscription price because you're monetizing it very directly — you are trading on this information, or your company is making different strategic decisions based on this new information you're getting. And that's one market. And then another one is that it's a barter economy where people distribute this information by trading favors. And this seems to be a lot of how news in DC works — people will strategically leak things that they would like to appear in the publication that they have a relationship with, and they leak things in a way that makes those things look better or worse for one side. It's just kind of a perk of the job — it's a little bit better than stealing office supplies, a little bit worse financially than embezzling money, but it is a way that you get to use your employer resources to accomplish goals that are not quite your employer's stated goals, and it's just tacitly understood that this is something people do.

The one of the problems with that is that when that information leaks out, it often loses a lot of its value once it actually gets published. So there is a quasi-news industry that consists of people who are gathering information about things that are happening and are not publishing it, and are taking advantage of it in various ways — one of which is to get it published somewhere in the way that is maximally beneficial to them, maximally damaging to their enemies, etc. So you can also think of — if that's one end of the spectrum, the other end of the spectrum is news that you read not because you want a differentiated view, but because you want to make sure you do actually have an identical view to your peers. And this is one of the functions of an op-ed page or of political endorsements or even of just the selection of what stories are front-page news. So if the New York Times were taken over by super right-wing people but they all had the same staff — these people are just not going to publish radically right-wing things — you could achieve a fairly right-wing aim by saying, okay, we're not going to tell you how to cover things, but the main things that we think our audience really cares about are crime and immigration and terrible red tape snarls, etc. You could have the same set of writers who are still engaged in the same process and still just by making those issues more salient — even if the coverage of those issues is actually tilted in a different partisan direction — those are just issues where Republicans tend to win when those are the issues people talk about.

Similarly, if you wanted the Wall Street Journal to be a tool of the progressives and didn't want to have wholesale turnover of the journalists, just tell them we're reporting a lot more on healthcare and we're reporting a lot more on economic inequality, and when we write about crypto and meme stocks we're going to cover the individual investors who participated in these. And you don't have to tell them, hey, you need to cover the individual investors who lost a lot of money, because if you say we should do a story on what it's like to bet on sports game outcomes and what it's like to put a lot of your retirement savings into Pump.fun, etc., you're naturally going to find that almost everybody lost money on it. So just choosing which topics are salient is an implicitly partisan decision. So yeah, I think that covers it for the news stuff. But yeah, news — it's a fascinating... Oh, there is one last piece.

So the last piece on this is: what is it that your readers actually want to get from a news story? So if you are covering Trump and your audience is mostly Republicans and you have a headline about a thing Donald Trump did, what is it that you want your readers to get from that? Now one view is that you want your readers to be very well-calibrated — if Trump did something good, you want to make sure they know it; if Trump did something bad, you want to make sure they know that. The problem is they don't actually want that. They don't want to hear this nuanced view where they used to be 70% on Trump's side and now it's 65% — that actually, to their peers who are going to be also selected very much on political agreement, that looks like backsliding. It looks like next week you'll say you wish you voted for Harris and the week after that you'll be a revolutionary communist. What people actually want is to basically maintain their views, and so you don't actually have a lot of upside from reporting on things in a nuanced and complicated way. You do have a lot of upside from, one, highlighting actually good things that your side did, and two, highlighting dumb things that the other side said about dumb things that your side did. So that does allow people to have this media environment where they say, okay, the president's a complicated man, but I notice all the coverage of him is unfair — which yeah, if you only read Fox and The Blaze, you are definitely going to get that view. And you can precisely invert that. What ends up happening is there are a lot of people who are pretty bewildered that their side is not the overwhelming favorite and they feel kind of under siege. But if they read the other side's equally partisan media, they'll be reading something from the perspective of someone who feels exactly as under siege, just from the opposite direction.

Yeah, so this is a fun one because search tends to assume that you know roughly what you're looking for. Even if what you're looking for is something like 'best place to get affordable vegan lunch in Midtown,' you still implicitly are assuming that someone has actually collected the data to answer that exact question. But there is this other kind of search which is a little bit more open-ended and a little bit more scope, which is a reverse search — where you're trying to identify something based on some known characteristics. And one case of that is reverse image search, where you see a picture, you want to know what it's from. And if it is from a movie or TV show and it's a popular picture — you've seen it 50 times, you just don't understand what the context is — you can usually find out. If it's a picture of a random person, it is very, very hard to get a tool that allows you to do that unless you're in law enforcement and subscribing to the right face-search products. Because that is a case where very, very few entities that have the technical sophistication to do that are willing to tolerate the risk and blowback of being able to de-anonymize someone based on taking a photo of a stranger in a public place.

And what I've been thinking about was that LLMs would actually be really good for a sort of qualitative reverse search — where you're asking who is the best person to talk to on topic X. And the more I thought about that, the more I realized that a lot of — there are various sites that are implicitly building the infrastructure to do that. So LinkedIn, for example — you could imagine a world where LinkedIn on the hiring side has a chat-like interface, and you tell it that you are a YC-funded startup and here's what you're building...

And here's how it's going, here's where we're located, and here's what our team is like. Please tell me who the best — like who are the 10 best people I should talk to about being our new CTO. And that would be a really cool feature, but it would also have this weird privacy-violating thing, which is basically that the person who gets the inbound request to talk to this YC company founder — what they know is that some mysterious black box machine just totally de-anonymized them as the specific person who is good for this role. And that's not necessarily something they asked for, not something they opted into, but here they are. So it is kind of disconcerting. It's disconcerting in a lot of cases.

So you could imagine — I'm sure that given the right data set, there could exist an LLM where I tell it, 'Here's where I live, I'd like to know who on my block, who in my general neighborhood I'd get along with best.' I'd like to meet my neighbors, but it'd be really, really weird to have someone knock on your door someday and say, 'Hey, an AI told me that we would actually be really good friends and I live just down the block, let's chat.' It is kind of a weird world, but it is a world we are gradually, gradually approaching because there is just more and more data in the training corpus.

So I do of course do the vanity Googling equivalent in LLMs. It is another nice way to test out models. And what the original ChatGPT did, which was kind of funny, was it basically inferred a really flattering background for me based on what I do now and then just kind of made up a bunch of stuff. Like I think I had an MBA from Columbia and had co-founded a private equity firm and all kinds of cool stuff. It was great. But now it's actually realistic and accurate, and that's a little bit disappointing but also impressive.

So yeah, I thought that concept of LLM-based reverse people search is an interesting one, and it's probably one where what will actually happen is that a lot of the companies that have implicitly created a really good database for doing this kind of search — they are going to monetize it either by building their own AI tools or by licensing all of their data to somebody else. And I'm sure that things like travel review sites could do a really, really good job. If you've got — so let's say there's Google on one end and they know a lot about your personal interests, and then there's TripAdvisor and it has all this data about what other people have said about different hotels. And Google can also look at its information on these people and figure out, okay, how similar is this person to you and how well do their taste predict your taste? And it could actually give you a pretty nicely summarized list of places to go, things to see, hotels to stay in, etc.

That would be pretty valuable. But I think it'll be interesting to see how much of that value gets captured by the companies that are building the model, how much gets captured by the companies that have been accidentally hoarding really valuable data around this, and then how much just doesn't get captured because the bid-ask spread is too wide for those to — yeah.