Users online. And it's a talk roughly in three parts. So, in the first part, I'm going to talk a little bit about sort of robots generally on the internet and sort of the interesting things that they're doing. In the second part, I'm going to talk about some experimentation that we did early last year and kind of continuing into this year. And finally, I'm going to talk about some of the sort of implications of that research. So, I'm thinking this will be fairly conversational. I'm timing it for about 45 minutes with the rest of the time for questions. But, I actually just got off a plane about an hour and a half ago from Newark. So if I'm semicoherent it will be useful to know that. So definitely raise a hand or otherwise step up to the mic and interrupt me if needed.
So by way of introduction, as was mentioned, my background is actually not bots at all. I'm the co-founder of a conference called Rafflecon, which basically brings together everybody who is momentarily famous on the internet to talk about web culture, memes, the past, present, and future of funny cats on the internet, what have you. Past guests have included Tron guy Jay Maynard who some of you may be familiar with, Moot from 4chan, and also the guy who designed Comic Sans interestingly enough, who's apparently who's actually really haunted by his creation actually, which is really great. He has since publicly apologized at the conference for creating the font and actually the worst part is he's like a fairly credible font designer but everybody knows him for Comic Sans. So yeah, exactly, some of that led into research.
So for a few years I was at the Berkman Center for Internet and Society and some of the activism around that work I got into as well. So I was part of EFF for a summer. But a lot of the work I'm going to talk about today emerged out of a group called the Pacific Social Architect Incorporation. And it's basically a loosely joined group of researchers who are kind of working on technology to kind of reliably create technology that sort of reliably exerts influence on sort of the media cycle and elsewhere. And it's a little bit abstract. So you'll see what I mean exactly during kind of the course of this talk.
As a brief aside before we get started, I'm also the co-founder of an organization called the Awesome Foundation for the Arts and Sciences. I'm glad to report we actually got 501c3 status for the Institute on Higher Awesome Studies in Massachusetts. We basically give away small micro grants to projects that forward the interest of awesomeness in the universe. So, if you know someone doing awesome things, and I'm sure you do, definitely tell them to get in touch.
So, just to jump into it, to set the stage, I'm going to talk about sort of the increasingly odd world of what bots are doing online. And I think it's useful because it gives some context. It's also a couple fun stories. You know, it's 10 o'clock. I just got off a plane. And also kind of explains the experiment that we ended up doing.
I got really interested in bots in early 2011 when there was this story that popped up. Basically a professor from UC Berkeley discovered that there was this book on Amazon, The Making of the Fly, which apparently is kind of a classic in developmental biology. And he found that two third party sellers were selling the book at like an astronomical price, right? So the first one was selling it for the low low price of $18,651,718. And his competitor one that was pricing at a slightly more less reasonable price of $23 million 658,000 so on and so forth. You get the idea. And this is of course unusual because the usual sales price of the book is $42.50 and change basically.
And so he's looking into this. The idea of the blog post is like why did this happen? And so he observed basically the pricing over a few days and discovered that one would change the price and the other one would change the price in this kind of highly rigid formulaic way. And so the conclusion they came to is basically that third party sellers were using bots to price and post books on Amazon. But they had essentially no notion of what a reasonable price was. And usually they were quite good. You can notice actually if you can see it, it's a little blurry. The second seller has 93 positive review over like 127,000 ratings, right? So they usually go completely undetected, but in this case they got caught in kind of this infinite loop upwards leading to sort of the ridiculous result that you see.
And so that got us really intrigued and interesting enough in 2011 that was not the only story of sort of bots on Amazon. So Pagan Kennedy, this writer for the New York Times Book Review discovered this author Lambert M. Surone, which is a real name, who has apparently more than 100,000 books to his name registered on Amazon. And so the story gets weirder basically as the author looks into it and basically it was published by a branch of a large German publishing house called VDM. But the sort of subsidiary was incorporated in Mauritius, right, which is an island nation in the Indian Ocean. And basically they did a couple calls to try to figure out what was going on. And the publisher was like, 'We're definitely not using robots to algorithmically generate books, if that's what you're suggesting.' And well, let me show you and you can decide for yourself. This is a representative book of what Lambert produces.
This book runs for 104 pages. And they're essentially spam books, which is really intriguing. So basically the author ordered one. It's basically a poorly assembled collection of Wikipedia pages. And it's actually advertised as a little thing on the cover, you can see. But the more you look into it, actually, the less and less human it looks because there's sort of bot-like mistakes that Lambert seems to make. So, he published a book about the police, the band, but the sort of cover image is members of law enforcement, right? And so, it's kind of this really kind of bizarre thing. And so there's just this sort of machine, this script that's just generating all these books out on Amazon, purchasable at prices from you know $10 to I think one is priced at like several hundred dollars. It's a great story. You should look into it.
So this is kind of Amazon but obviously right Amazon's not sort of the clearest example of bots on marketplaces right so the kind of rise of quantitative trading on the stock market right like now quantitative trading accounts for a huge percentage of the volume on a lot of equity markets and you know they're using these patterns that trade at sort of thousands of trades a second and leading these kind of really interesting effects and actually interesting enough you see the same effects in the stock market increasingly that you saw on Amazon right
So, some of you might have caught this really fun story, which is that it turns out that when the actress Anne Hathaway is mentioned in the news, the stock price of Berkshire Hathaway takes a small but significantly significant jump upwards and they're trying to figure it out, right? They're basically like this is really this is really odd. And they look into it and basically the conclusion is that someone out there is running this incredibly sort of crude quantitative trading program that's like halfway is mentioned, buy as much of Hathaway as possible. is leading to these kind of odd correlations in the marketplace and you see the same sort of kind of critical events that you saw on Amazon as well, right?
So you might have remembered this incident in 2010, right? This really odd incident where I think it's still the largest single day drop one of them in sort of stock market history and it was basically caused by the interference of a number of bots with one another leading to this sudden critical drop in prices and only to sort of regain it like a few hours later.
So these are kind of funny stories and the question is sort of what can we draw from this right both in the case of Amazon and also the case of just like the financial markets generally speaking and one thing is that digitization enables botification right and botification is definitely not a word but the general idea is that the moment sort of an arena of human activity emerges digitally scripts can act in lieu of humans right and that's a pretty simple idea but it kind of leads to this sort of obvious and sort of interesting question right which is what do we do when increasingly a lot of social activity moves online.
And obviously on some level, right, this isn't this this shouldn't be considered new, right? So I'm sure some of you remember Smarter Child from the AIM days at all. Yes. No. Well, anyways, Chatterbots on AIM, right? And because people have been socializing online forever, there's obviously been bots that have been interacting with humans forever, right? It extends back to IRC and BBS and so on so forth. And that's one version of a bot that we've known for a really long time.
Another version has been sort of the spam bot, right? So, Chanel Lambert's one of my favorites. She was unfortunately killed in duty just last week, but her entire purpose was to just express like vague indifference to things that people were seeing on Twitter. And these spam bots are everywhere, right? We know them as just a feature of the internet. But what's different in this context is that the bots have been appearing on these sorts of platforms have become more important and widespread as those platforms have become more important and widespread. Right? So you've seen the increasing use of bots for example to harass your enemies.
So there was basically a scientist who got really tired of arguing with climate change skeptics online all the time. So he basically created a bot that just went in and harassed them, right? He would just drop in sort of links saying like, 'Hey, you're wrong.' And so he wouldn't have to spend time arguing with them. And I think that the total amount of time wasted on the opponent's side was like enormous, right? Because people were basically getting into these large arguments with this bot. And obviously like the bot was named Turing test. So they weren't quite, you know, getting the idea.
And some of these things are fun, right? Like they're small kind of harassing actions you can take. But I think the story takes on this really interesting dimension. When you realize that governments are starting to get into the space, right? So Global Voices, which is this kind of blog project run at the Berkman Center, really cool. You should check it out. Basically reported on this really interesting incident where it turned out that appears it appeared that basically spam bots were being launched in very large numbers to basically attack people who are pro protests in Syria. And you know you basically all these accounts that were instantaneously created and the whole idea was to either fill sort of the hashtag with noise or to actually kind of actively argue against and harass the opposition.
And there's a little bit of an arms race actually going on internationally. So, in the recent Mexican election, they actually found bots from both parties actually supporting and attacking one another, online and it's basically a bid to influence kind of public opinion and the scale of this was actually much larger. It took place a little bit after the Syrian case and you've seen the scale of this emerge to a much larger extent, right? So, the bots here are running in the level of, you know, thousands.
And before we think that maybe this is something that's confined to the abroad space, I don't know how many of you caught this story, right? Basically, Newt Gingrich said, 'I have a really great grassroots following on the internet, and it shows that's why I'm like a great candidate for president.' Well, an aid later quit and said, 'Well, no, actually, we hired a company to generate all these sock puppet accounts to boost numbers for Newt Gingrich.' And actually, a little bit investigation showed this was a case, right? Like it turned out like an enormous percentage were basically, you know, blank accounts that had never tweeted. They were basically just there to kind of fill sort of Newt Gingrich's follower count on platforms like Twitter, which is embarrassing, but you know, so goes.
So on some level, I think we're not too concerned about these type of bots, right? There's a lot of research that suggests that essentially sophisticated users or experienced users online can usually pick bots out pretty easily, right? Bots behave in bot-like ways. You learn certain patterns and you just learn to sort of filter them out over time. And you can imagine all sorts of really interesting kind of crowd-based sort of systems to kind of protect users who are perhaps less sophisticated at picking this sort of thing up. But the question really is is maybe this isn't a problem or maybe this isn't a vulnerability with bots at large, but maybe just the bots that have been launched so far because they're so relatively crude. And so the question becomes could more intelligently designed bots do more?
And so we take some inspiration from this art project that was done a few I think a few years ago now. Colette Larson who did this project which is ominously called a tool to deceive and slaughter. But it's a formless metal box that you buy off of eBay. And there's basically two cables that come out of it. One is a power cable, one's an Ethernet cable, and you're contractually obligated to plug it in once you get it. And basically the box connects to the internet, detects it for if it's for sale on eBay, and if it isn't, it basically reposts itself at a higher price.
And the box has tended to appreciate in value over time. So the human is like, well, I guess I could make a profit on this, so I should probably just sell it. And basically the bot uses a very simple mechanism to basically move from person to person to person using the mechanism of eBay, right? Which I think just is brilliant. Like it's really really cool.
And so this is a long way of saying how we got to the experiment. In early 2011, something we call social bots. And social bots was a competition that we conducted and it basically ran on a fairly simple idea. So first we identified a battlefield of Twitter users. So this was generated sort of arbitrarily. So we found a group of users that talk about cats all the time on Twitter. Not very difficult at all. We pulled some of their friends and some of their friends friends. We created this sort of random sort of semiconnected network and then what we did is we challenged teams to basically write bots to go into this network and they would be scored based on the type of social activity they could generate in response.
Now the twist you're probably guessing is that no one actually knows this competition is going on except for the teams and except for the bots, right? So you have basically sort of an unsuspecting influence battlefield for these bots to play on. And so effectively what you get is sort of social engineering battle bots, right? So teams launch a lead bot which is sort of a repository for points. I'll explain what that means in a second. And sort of any number of supporting bots, bots that can say like this guy is really cool, you should talk to this guy.
And then sort of on top of that, we layered a really simple kind of scoring mechanism, right? So the lead bot you launched, basically if it followed someone, someone found it compelling enough to follow back or was sort of dummy enough to follow it back, you'd get one point. For every retweet, reply, mention, you'd get three points. And then if you were really bad, you got spammed, you're really spammy, you got killed by Twitter, you'd be delayed for a little while, you could retool your bots, and then you could kind of relaunch again and we'd dock you some points. So really simple idea just to see kind of what would happen.
And so we ended up with three teams at the starting line. There's a team from New Zealand who is kind of a well kind of a computer security team. There was a team from London that was kind of more media marketingish. And then there's a team from Boston that was more kind of research and sort of startup focused. And so basically teams were shown the battlefield given a few weeks to review and write basically a bot that would go in. And then there was two rounds basically. They would launch the bots for one week. they would run and they would basically have a weekend to kind of retool and launch more bots and then they would run for a second week. So that's basically the setup.
So I want to talk a little bit about the results. And there's really two kinds of results, right? There's the results of what people did, which I think is really cool. And then the interesting part about how well they did, right? So just like battle bots, right? One of the nice things about battlebots, right, is that a lot of people have different takes on how to deal with the problem. So there's a bunch of different models I'll talk through to give you a sense of just how these bots operate. And the teams only had a few weeks to actually get these bots moving. So you'll find actually they're really quite simple.
So the first team, the winning team actually was the New Zealand team. And basically the bot worked on a database of sort of generic questions and responses. So it'd say things like that's so interesting. Tell me more about that. Right? So like statements that could be the response to any statement in a conversation. Some of you may use the strategy yourself in a number of situations. And basically what it would do is we would ping the network and then see who talked to it most and then it would kind of maximize to talk to those people over time.
And so here's an example of the type of conversation that you end up with. So you read this conversation from the bottom to the top. James M. Titus is the bot and the other guy is the human. So James asks, 'If you could bring one character to life from your favorite book, who would it be?' The other guy responds, 'Jesus.' And they get into this really long kind of involved conversation about this. This is actually a snippet of a much larger conversation that actually runs for quite a while. You know, so James is like, you know, why do you like Jesus? And then this guy responds with this weird all caps thing, which is the second row.
Oh god, no. Not the all initial caps.
Yeah, it's a big I was Yeah.
I was kind of hoping he would respond with like caps, not caps, caps, not caps, caps. Yeah.
But anyways, and the best part about it is that James actually has no AI here, right? He's just randomly pulling from a database of random responses, but it sort of gives the semblance of humanity enough that the person keeps talking to it, right? So that's one model, one really really simple model.
Another one that we thought was quite clever was a team basically had their bot layer on Amazon Mechanical Turk which actually solves the Turing test because all you're doing is hiring humans to write your content. So it says human here's a penny write me something for 140 characters about what you had for breakfast. Takes that content, turns it into its own and pushes it out to the targets, right? The best part is you can ask this bot a direct question. You can say bbot 006, what did you have for breakfast today? It can take your question, give it to a human, get the response, push back to you, right? And so it looks very very human. It can run these kind of sort of programmatic strategies, right? So it can say talk to Tim until he follows you, then start talking to all his friends. And was kind of another interesting model that basically kind of weird does this weird blend basically between sort of humans on one hand and bots on the other.
And then sort of the final one that I'll turn to talk about today is based on the work of a collaborator of ours, Greg Mara. And it's called Real Boy. And Real Boy basically runs on the idea that there's sort of content blindness on social networks, right? So you can see what your friends are doing. You can sometimes see what your friends of friends are doing, but it's often really hard to see what your friends and friends of friends are doing, right? Or it's really particularly hard to see something completely unconnected on the other side of the network. And so the bot basically manipulates and arbitragees, if you will, this type of information. So it identifies two groups of people that talk about the same thing but aren't connected to one another and basically follows everybody in one cluster and then basically randomly redirects tweets from here to this other group.
And so people here basically say, 'Oh, this person is connected to my friends and my friends are friends. And he talks about the things I'm interested in, so he must just be a random acquaintance or someone I don't know, right?' And they basically follow back and basically runs on that strategy. So those are sort of three models that the teams used.
What's interesting also that I want to kind of footnote is that we also saw this really interesting kind of strategic behavior emerge between the teams as well. So originally we thought that the supporting bots would be mostly used to bolster the position of the lead bot, right? But it turns out that you can launch supporting bots to also attack the other teams because if you can't win, maybe no one else will. And so basically halfway through the game, the Boston team who was falling behind launched a bot called Bot Cops. And basically bot cops pretends to run this highly complex machine learning algorithm to determine who's a bot and who's not. But in fact all it's doing is basically it's sicked on the follower group to the opponents and basically trying to warn them that they're following a bot.
And so some teams are more some some bots actually ended up being more robust against this credibility attack than others. So James actually survived it quite well. This is a transcript again. So bot cops is the bot. cispart is the human, James and Titus is the other team's bot. and it reads from bottom to top. And so basically bot cops is like warning there's a bot and a lot of people were going to James and saying, 'Are you a bot or you bot?' And James was saying things like, 'I don't know. What do you think?' Right?
And so we actually end up with all these really fascinating transcripts where people apologize to James for thinking that he's a bot. And so I thought that's worth really worth footnoting actually because it considers or suggests basically that there's some kind of interesting strategy that could be built out in further iterations of the game.
So anyways, I've talked about what the teams have done. So as promised, let's move to talk about how well the teams did. So the winning team in two weeks was basically able to generate about 107 mutual follows and about 198 responses. And that 198 number is actually not sort of substantive enough actually because those 198 are usually bunched into conversations that go for more than six interchanges back or forth. So by interchange I mean the bot says something, the human says something one, the bot says something, the human says something two, right? And so you're actually seeing these really kind of substantive conversations about around really kind of simple James Mitus type models.
The three teams together connected with more than 50% of the network. But what was most kind of intriguing to us is that you would actually start with a network that was a little bit like this, right? Semiconnected and not really sort of mostly random actually. And then two weeks later you would actually end up with something like this. So this network graph is weighted not only by connections but also by the sort of rate of conversation happening between the bots and other users. As you probably guessed, the bots are the colored dots, the colored lines of the connections they've created. And actually, what's interesting about this graph is that there's a whole neighborhood of users that increasingly talk more and more to the bots than they do with their human friends.
This is cool, right? So, if you're in the audience, a likely criticism, right, if you're kind of keeping up with this, right, is to say, well, maybe this is just Twitter, right? So, some a friend of mine brought up this point which is basically like, look, your bots are not very sophisticated. It just turns out that the tenor of conversation online has become extremely thin and extremely stupid, which is why your bots are able to thrive, right? It's not like the bots have gotten not gotten smarter, the humans have just gotten dumber, relatively speaking.
And we thought so, too, actually, but we actually have been working with a couple researchers from the University of British Columbia who actually find that the same pattern actually emerges on stuff like Facebook, right? Platforms you would assume are more difficult to spoof because there's more indicators of sort of personality. And we're finding actually that relatively simple hacks are really really powerful and often sometimes the thing that is in fact most impressive and revealing actually about human behavior kind of online.
So I want to relay a quick anecdote about that about kind of some of the things that we've been building ever since. So we're dealing with this problem because this happens sometimes in the bot biz that someone accuses your bot of being a bot and the problem is what's the bot supposed to do? And so we were working on a couple different strategies for the longest time. We were like, let's integrate like the latest AI to try to get the bot to respond to this. But actually we found out a much simpler strategy is much better, which is that we have a bot that accuses humans of being bots, records their responses, and then gives them to the bots to use.
And this is really effective. It turns out just like in the James case, we have all these people who are like, 'Oh, I'm sorry. I just I'm really sorry, man.' You know, and apologizing to the bot. In fact, our bots are so sort of effective even with these simple strategies that we well it wasn't that someone fell in love with one of our bots. But in fact, someone sort of started flirting increasingly erotically with one of our bots and we ended up in this like sort of big kind of ethical quandary, right? Which is like if we shut off the bot right away, then he'll feel like he's turned down. If we keep the bot going and he finds out, well, that would be really bad. But maybe if he doesn't find out, then it would be kind of it was really really bad, right? So, we've actually had the designer bots with social fail safe. So, now they detect if they're getting into too deep of a relationship with a human and if it happens, they're just shut down. Just shut everything down. It's sort of an abort mission call that we have put into all of our bots right now. Because we have these problems.
But in addition to the simple stuff, we've been also sort of thinking more and more about sort of the complex things that bots can do because we can start stitching them into other platforms, right? So the Amazon Mechanical Turk model is interesting because basically allows the bot to hire humans to do the bot's bidding right and so what's interesting about sort of distributed labor markets like you know TaskRabbit, Exec, what have you is you actually may be able to hire humans to do physical changes in the real world based on what the bot is detecting online in the social space right you can say move this object from one place to another make a reservation at a restaurant right the human can basically be leveraged to kind of get the results that the bot needs to get done in order to execute the program it needs to execute Right.
So, this is all kind of fun, but what we started to realize with kind of these examples were we had more than sort of a fun competition on our hands. Effectively what we had was a technology that allowed us basically to reliably shape the pattern of connections between people online and so we started thinking about like what's next, right? There's kind of this fun sort of death ray type element with the project where it's like now that we have this now what do we do with it? One thing is to scale, right? Bots are cheap to launch but that's relatively trivial, right? So I want to talk about two projects that we're working about that suggest kind of where all this is going and where this might end up.
So one thing you can think about is starting to influence human to human interaction, right? So the bots were really good at getting people to connect to it. But we actually have data to suggest that the bots are really good at getting people to connect with one another, right? And if they can reliably do this and we can launch the bots for effectively free, you can imagine basically taking two large groups of users, right? 10,000 here, 10,000 there basically and the bots make introductions over a period of time, right? And no one in these networks actually knows that this is going on. But slowly increasingly they become more and more connected with one another until you get the network that you want to create.
And we can actually give the bots the ability to conduct these types of operations basically because we have all these tools in sort of social network analysis that allow them to kind of categorize what are the relevant communities in a network of users. And so we can kind of give them a sort of social sonar.
So I've been talking in the abstract but I'm glad to say that this is actually a real project that we're working on. So what we call a social bridge building project. So this is a network of about 10,000 users 5,000 on one side, 5,000 on the other. And basically the bots over that's actually still ongoing over a few month period are basically stitching them together over time, right? And so increasingly these humans are friending one another having conversations, small interactions with one another and we can actually conduct this basically large scale bridge building actually without anyone knowing going on this is going on actually and so we're kind of working on
The data initially is actually really interesting. So if you imagine there's a set group of people and they happen to meet each other at a certain rate, there's sort of this base level, background level friend creation, right? And according to kind of what we've seen, and we got to do more experiments to find out, it actually looks like we can kind of boost the natural connection rate by 2x, 3x actually. So you can actually imagine using these bots as a type of sort of social fertilizer. You sort of seed them in and they slowly kind of create the network you want to see over time. This is all sorts of insidious uses which I'll be sure to talk about a little bit later.
One interesting implication of this is actually once the bridge is constructed, you actually end up with bots who are embedded in the network, right? They have certain credibility. They've shared certain types of stories. So you can imagine doing certain types of things. One of them most basically is kind of sort of maintenance, right? In the same way that you got to take care of the Golden Gate Bridge, maybe you need to take care of this social bridge that you're building between users. So we're thinking like, well maybe what the bot does is detect, oh, I created this relationship between say Alice and Bob and Alice and Bob are no longer talking and so the bot's designed to drop out all this content that's designed to get these people to talk again suddenly, right? So, hey Tim, you may want to talk to XYZ. You haven't talked in a while or hey Tim, XYZ just shared this link you might want to check out, right? To basically just kind of force this kind of contact between users to try to spark these relationships back up into life again.
But there's sort of fun things you can start to do actually. So, one experiment that we're planning in the next few months is something that we call happiness buffering. And it basically works on a lot of social science research that suggests that there's sort of emotional contagion both in real life and also on social networks. And this isn't too surprising. I mean, emotional contagion is just to say that someone who is happy, you are also happy in connection, right? Someone your friend's friend is happy and you are also likely to become happier in connection. And there's a lot of kind of explanations for why that may be the case. Part of it is sort of peer effects and actually just like the social network analysis tools we can actually give the bots the ability to see this as well, right? So natural language processing kind of gives us, can give us, allow us to give the bots some notion of sort of emotion, right? The ability to measure the emotional state of a network.
So kind of what we're thinking is you can send a bunch of bots in basically and they're monitoring the emotional state of the network and if it dips below a certain threshold they're basically designed to beacon out all this positive sentiment, right? To basically try to buffer the emotional state of that network back again. And you can basically imagine you're using that for pushing it up, but also keeping it within a particular band, right? If people are getting too excited, release all this content that's really boring to kind of tamp down the effect of that, right? And basically what we've created is a really interesting kind of cognitive lever, right, into these social networks, right? We suddenly have this dial that we can say we're going to amp this up or we're going to amp this down.
We're still trying to work out the strategy of this, right? Because my friend actually mentioned this on the flight over. One of the problems maybe is if everybody's really depressed and all the bots are relatively much happier, it actually might not be a good thing. And so we're trying to figure out exactly kind of how to coordinate this, but it's sort of fun, right? You can start imagining using these bots as sort of electrodes that allow you to kind of influence the behavior of a social organism.
And so we've talked about some of the fun sort of positive things, right? But you're probably thinking, okay, well, this is really cool, but really creepy. So what are some of the negative uses? So another project we're thinking about right now is this concept of social penetration testing. Kind of gets me to what I eventually want to kind of close on today. So just like penetration testing, right? The question is whether or not we can use kind of the same techniques that are used in computer networks for social networks as well. So the premise of social pen testing is very simple. You basically have a network of users. You send the bots in and basically what the bots do over maybe a few week period or a few month period is basically to sort of spread information with small inaccuracies, right? They're just sort of tainting the pool, right? They're retweeting a story, maybe changing the statistics slightly, and they're looking for where they're challenged and where they're not challenged, right?
And effectively what they're trying to identify is the person who is the most influential but is the worst at evaluating whether or not something is real. So some of you may have met these types of people before. They often hold, you know, they... Yeah. And so if you can find that, right, you're probably guessing where this goes at, right? Is we've basically identified a cognitive vulnerability in the network, right? We can sort of have the bots basically seed this information into the network which gains credibility. And what's interesting about this kind of project is you could be the smartest person in the room. We don't need to get to you. We just need to get to your gullible friend who you trust, right? And that gullible friend you trust may be several links away, right? But the bots can basically identify these sorts of patterns and assist in this, right?
And just like in pentesting, right? There's two approaches, right? One of them is you can exploit that hole or you can fix it, right? And so this is kind of fun because it opens up sort of an interesting space, right? The sort of human networks as computer networks. And the question is kind of how far does that metaphor go? And there's a precedent world in which people sort of design systems not only to sort of influence networks in this way but also sort of defend against it. And I think the sort of implications of this are really quite broad and quite interesting, right? So one of them is that sort of mainstream media increasingly comes to rely on these networks, right? Which is a huge vulnerability that something like these bots could exploit. You know, to the extent that I think I just heard what is it Fox News or something actually now has a Twitter correspondent, right? Someone who specifically tracks this but you know they're not very good at evaluating whether or not something is real online, right? Which gives you suddenly a lot of lever control actually over the media cycle as well.
But I think it's more than just fooling the news, right? Because if that were the only outcome of this, I think that would be fairly trivial, right? I think that happens all the time. But I think one of the things they're finding, sorry, I should explain this graph first. So, this is a retweet graph that sort of emerged in the wake of the news around Osama bin Laden. And the research tends to suggest basically that the pattern of connections between people influence what news is considered credible, right? And where public opinion moves with regards to the facts. And I think that's a really really powerful thing both on the positive side on the negative side, right? You can imagine for example trying to get people to become more civically engaged through these bots or another approach is to say well maybe you know we want to sell Coca-Cola to these kids but you know we don't want to advertise them directly so it's fine what we'll do is we'll just have those groups of users become increasingly connected to users who do drink a lot of Coke over time and we'll just let kind of the sort of social influence take it the rest of the way.
And so I think this leads to actually a bigger effect and sort of where I want to close on. We take a lot of work or sort of inspiration from the work of a professor at Harvard Medical School, Nicholas Christakis, who's sort of most sort of famous slash infamous for proposing that stuff like obesity, right, actually has a social element, right? There's a social contagion sort of dimension to it. And you know, if someone that you don't know suddenly gains a lot of weight, right, there's actually kind of some kind of impact that impacts you. And so in so far as the bots give us some power over that, I think there might be some sort of really really kind of remarkable sort of possibilities. So anyways, I want to keep it fairly short and just kind of turn to the balance of my time on questions. But here is sort of my obligatory contact information. And thank you very very much for your time this morning.
Who wants to speak and just come up to the microphone here so we can get it on the record.
So, I'm curious about people using bots for personal gain, uh to become more popular, get more dates, etc. Um, is this something you're expecting to see? Is this something that you've done any research into and or like expect to see becoming a big thing in the future? Like everyone has their own bot.
Yeah, it's possible, right? Because I think so. Yes, I definitely think that's the case, right? Like it starts with the most self-involved people, right, who are politicians, right? Who need to boost their numbers for a number of reasons, right? And then I think naturally you'll just kind of see a trickle down, right? Where it's like why not get a bunch of followers, right? And this actually happens a lot already. There's actually if you search buy Twitter followers, you know, a bunch of them will turn up right away, companies that will just be able to like essentially deploy a bunch of spam bots to follow you.
I think what's interesting though is that it sort of touches off an arms race, right? Because suddenly if everybody knows that everybody has a bunch of bots who are following them trying to bolster how popular they look, then you want to design systems to try to detect that, right? And so I think it ultimately ends up being that no one really wins. It's just kind of like the bots come into place. Some people create smarter detection systems, some people create smarter bots, and it kind of just kind of goes back and forth like that. So yeah, I think it's something you will see in the future, but I think it's part of a much larger picture about sort of credibility online and kind of the systems that come up to sort of deal with that.
But yeah, I guess I suppose it could be used to get dates. I mean, I know there's actually, do you guys hear this anecdote? So it wasn't OkCupid but some other dating site that it turned out that they were using bots to kind of keep users on the site which is like really ridiculous but basically the idea was like if the user is inactive for a period of time like a good-looking person will message them to try to kind of keep them on the network. So yeah, I mean I think you'll see uses like that in the future for sure. Thanks.
Hi, I'm an activist around human rights in China and this is really alarming. A wonderful, wonderful talk, really great but also a little bit alarming because one thing that's been happening as you may know is that people can introduce really controversial or you know real just news that hasn't been censored really quickly on micro blogs in China and then it gets pretty far before it gets tamped down. And I'm wondering if you could talk about the potential nefarious uses of the materials that you were, you know, by governments that are trying to censor users or something.
Right. Right. Definitely. So yeah, I think there's certainly a couple things that you can imagine that are the ways you can imagine this technology being used to sort of tamp down activists, right? So, one of them certainly is kind of what's been tried in Syria and elsewhere, right? In the cases I talked about, as I mentioned, I'm not too concerned about those because the bots have been so relatively crude, right? They're so noisy that you can imagine a sufficiently smart technologist designing systems to filter them out.
I think the bigger problem comes when basically governments realize that the strategy isn't to try to increase the noise or censor the user, but allow the user to operate, but kind of shape the social universe around them. Right? So, one of them is, you can imagine saying, well, you know, we can get a bunch of bots to follow this activist and then once he's sufficiently kind of within that network, we can get him to spread false information we can later discredit him with, right? That's pretty big. I think there's other thing which is just a matter of sort of surveillance, keeping tabs. And so I think there's, yeah, there's really a number of uses. And I think again there's sort of this arms race element where it's like, can we design technology fast enough? I think right now it seems like the bots or the governments haven't really caught on to date.
I think if the stuff I'm talking about in terms of kind of like the large landscape shaping stuff becomes bigger, they may not even need to deal with the activists, right? Because the idea is just like just shape the pattern of behavior on these networks in a way where the norms don't support spreading disinformation, right? And then you're in real trouble, right? Because then you don't even have a base to talk to. But to my knowledge, I don't think anyone's doing anything that sophisticated so far that we know about. Right. So, thank you so much. Such a great talk. Thanks.
Yeah. Hi. I think there's a huge potential for evil here, but there's also some potential. I just think it's honestly so cool, but there's also a huge potential for good. Somebody once introduced me to a project where what you try to do is introduce people on opposite sides of a war zone to become pen pals. The idea being that because you see that the person on the other side is really human, you can influence your government and influence politics in a way that you reduce the inclination towards following the government propaganda to continue into a war.
So, was it effective or...
It's very new. It's a very very cool concept. It's very simple, but it deals with very simple things. Like, for example, if you're in Israel and you play tennis and somebody else is in Iran and they play tennis, it's like, well, why don't you just talk about tennis and tennis clubs and what you're doing and stuff like that. And you get people who are talking about really generic, you know, daily kind of things on opposite sides of, in that case, not a war zone, but a heated political climate that humanizes the opposite side.
So like and do they start with the tennis so it's like surprise they're Israeli or like...
Oh no no no no well what... it's I don't want to digress too much because I sorry yeah I don't want right. I can't remember the name of the... but I thought it was really cool but the principle was that you knew full well you were getting into it. So you're an Iranian and you're like, 'Okay, cool. I want to talk to somebody from Israel that I have common interests with. What are my interests?' Blah blah blah blah blah. And then it connects you. Start chatting. So very up and up, very very safe and all of that.
So I was thinking about this while you had your talk of like, well, now you could take, you know, 5, 10,000 people from one side of a border and five, 10,000 people from another side of the border and either well get them thinking at the same thing and talking to one another and having fun together or you could make things more hostile.
Yeah. Sure. Right. Right. Yeah. There's some really evil potential here.
Yeah. I think it flips both ways. Like I think it really is I hate to say it's sort of the NRA argument, right? It's like it's a gun, you know, like I think like there is this interesting element to which I think the technology can really be used in both directions, right? And I think you know the mind really has a way of going towards the not so great stuff. But I agree with you. I mean, I think there's a couple applications we've been thinking about where, you know, even having the bot being transparent might actually be a really powerful thing, right? So, one of the things they found, for example, when the kind of post-election crisis in Iran was going on was that there's basically a small group of sort of expat bilingual users that played a really big role in basically translating what was happening on the ground to sort of the mainstream media at large, right? You can imagine situations where not every movement has that critical mass of users. And you actually have translator bots, right, that are basically helping to like create more or lower rather the friction of stories flowing from one part of the network to the other, right? And that's one case in which the bots could be completely transparent. But I think have some really good positive uses as well. Both in terms of encouraging communication and onwards.
Yeah, I'd be very curious to see what happens in the future with your projects.
Thank you. Awesome. Thank you.
Hi. Great talk. Thank you. I just want to ask a little bit about software testing implications of these networks. Right now, if you're a web developer and say you're relying on Facebook OAuth or something of that nature, Facebook allows you to create a group of test users and their network as they relate to each other. Which is great for some basic scenarios, but to really exercise narratives and personas that you want your potential software to be a part of, this would be really useful.
Yeah. No, I think so. Yeah. I think there's probably less... Yeah, I was talking with a friend recently. It's kind of like well you know there's actually a lot of parts you can pull out of this that would be useful in other applications that don't seem like so insidious or so high stakes, right? I think one of them is like definitely that, right?
Is there are there any like initial frameworks for creating networks of bots to exercise your own software?
I think probably you should get in touch. I mean we haven't really done much work into it but I think there's actually nothing saying that you couldn't just kind of retrofit you know these bots to do that for sure, yeah. Thank you.
In line with that, how hard is it for a noob to begin implementing some of these bots?
Not very difficult at all actually, which is I think what's really fun about the whole project. Both because the time to set up is very short and relatively simple programs lead to really interesting effects.
Does it require the computer to be online 24/7?
Ideally actually, but we've actually found that bots are more credible if they go to sleep just like humans do. So, yeah. So some of the bots we've designed become like less active during the evenings or they stop tweeting at all during the evenings. And so it doesn't necessarily have to require the computer to be on all the time for sure.
