Panel Discussion: Managing technology hype in publishing companies moderated by David Smith, The IET

Okay, right, bear with us. Definitely hoping for some audience participation on this because I got slightly panicked text messaging and a phone call late yesterday. Obviously Tasha is at home with the plague, so we'll see how this one goes. What I'm going to do is just start this way. I don't know whether you talked about an order you were going to go in, but you're going Nick, Michael, Phil. That's how that's going to work right now. So we're going to start off with them introducing themselves, and I believe you have an amusing anecdote on this subject area that you can hold forth on. So introduction and anecdotes, and then we'll see. I've got one from where I work now, but my boss is in the room, so obviously definitely... but depending on what Phil's anecdote is, I might be able to give you another one from where I worked previously. So Nick, would you like to start? Tell us who you are and what your thoughts are.

Absolutely. So my name's Nick Kind, and I work as a strategy consultant for publishing companies, mostly in the education sector. I'm going to tell you a story about the education sector in a moment, but I've also worked extensively in scientific publishing, not least when I ran a strategy function for Hawksbrink, who owned Springer Nature amongst others. When Tasha asked me to do this, I think she wanted me to bring another perspective onto this, so I am going to talk about an education example. I currently work, as you will see, in fact one of the things I do is advise a company called Area Nine, who are an extremely successful education technology company working in the area of adaptive learning. The story that always comes to me when people mention technology hype—in fact there are two, I'll save the other one maybe for later—is all around the great white hope in education technology, which any of those of you who have been near the sector might have heard of, about what is sometimes called personalized learning and sometimes called adaptive learning. I believe there is a difference, where the belief is that the technology will allow us to deliver a deeply personalized learning experience one-to-one for every learner, and by doing that, not only to simulate all of the positive aspects of having a personal tutor—which is very much viewed, and the evidence seems to support the fact, as the best type of learning experience—but it also allows that to scale in an extraordinary way. About five years ago, there was a very well-known company in this space, who will remain nameless for this, although any of you who can use Google will be able to find it very fast, who raised well over a hundred million dollars as an adaptive learning provider. It got to the peak, the height, to a point where the chief executive of that particular company described—and I have to take my notebook out to get the words exactly right—described his solution as a mind-reading robot. No hype there. He also, by the way, and this is the best of all, described it as a magic pill, quoting that very moment in The Matrix where one of the main characters pops a pill and knows everything about everything. This is presented as a bitter pill for anybody else selling adaptive learning. The point being that I now have to walk into meetings to talk about Area Nine's adaptive learning solution, which has served millions of students in the US higher education community and thousands of doctors, by starting to say, by the way, this doesn't solve all of your problems. Having to go into a sales conversation by saying I'm just going to tell you what this doesn't do is actually quite an interesting dynamic. There was a moment where all of the publishing companies in the education sector were desperate to have an adaptive solution because it was the biggest buzzword in the arms race that was constantly going on between the different products in certain sectors, particularly in US higher education. I think managing that hype is really what we're here to talk about today. So I am not a mind-reading robot teacher, but I think what we're here to talk about is how you manage that hype and make sure it doesn't get out of control.

Okay, thank you very much. Thank you. I'm Michael Upshaw. I work for Seiler, which is a Danish machine learning company, but my background is in publishing. I have many years of experience on both sides. My anecdote is really back in history, and as with Nick, it's revealing a kind of symptom of the situation. Several years ago, I was working with a small publishing company—and this dates me—we did one of the first online encyclopedias in the UK, seven members many years ago. Lots of interest in the company, Microsoft bought a stake in it. We were doing very well, and remarkably we were bought by a well-known company on the high street, WH Smith. WH Smith's share price increased by significantly more than the total cost of the acquisition of our company. From that day to this, I don't think I've ever understood what on earth the relationship was between an online publishing company and a high street retailer of newspapers, stationery, and confectionery. But the hype was there, the win was that magic term 'online' in the analyst description. If we fast forward to 2018, there are at least two examples just this year of publishing companies that I work with where exactly the same thing has happened in the area of AI. One publisher announced their annual results, and the financial press said, 'Good company, excellent growth, but why isn't there a mention of AI in the results?' Exactly the same thing for me as a working with a producer of new technology. Such hype is irrelevant, unwanted, and really unhelpful.

Okay, thank you. Yeah, I've got both my previous boss and my current boss in the room, so I can't really say anything. So my example would be: we have CABI, which is biosciences agricultural sorts of organization, doing abstracting and indexing. We came up with a product idea called, or based on, biofuels—which might be the example. First off, we let our marketing team name the product, and if you want to write down 'biofuels exchange' and see what funny word comes up in there, you can't. So we had to change it to 'Biofuels Information Exchange.' But it was back in the day, this was early 2000s, 2010 possibly, when we were looking at controlled vocabulary methods and data and alike. CABI has a great taxonomy and thesaurus, and we gave biofuels information to a completely different organization who promised the latest and greatest in applying taxonomies, ontologies, and any other buzzwords you can come around that sort of thing. What we got back was our own information with MeSH doing it, rather than utilizing our own control vocabulary. All of a sudden we were publishing lots of things on alcoholism and alike when actually it was all about biofuels. So I suppose my main thing is really around lessons learned from that example. That's very much about know what you're trying to seek out, have clear goals of what you want your new technologies to achieve, and it's not just about the technology for technology's sake. So it's always time-box it, and also have some good scientific methodology of knowing what you want to achieve out of it.

Brilliant. All right, he went for exactly the same example as I was going to give you from my previous employ. One of the things actually about that particular example, I'll actually tie it into mine. Anybody here working with IBM Watson? I see those started sniggering right. So I think that's one of those classic examples. There is an organization that my organization knows dearly, and they have spent an awful lot of money historically working with IBM to try and get a version of Watson going for engineers. Again, I'm not going to name that organization, perhaps you might be able to figure it out if you were really dedicated. But the fact of the matter is they burned a very significant seven-figure sum indeed and had absolutely nothing to show for it. One of the things about that, as being someone that was on the receiving end of both that organization that Phil was referring to and Watson, was that they both gave really, really good demos. I think that's one of the things that my experience of this has been: there are some things that come out and they look like witchcraft, right? You're thinking, how can that possibly be true? The company that Phil is talking about no longer exists. They were called Calexis, I think Elsevier bought them, and I don't know what ended up happening with them. But it's great was their money. They gave an absolutely mind-blowing demonstration, no two ways about it. As an organization, the people that were there asked all the right questions of that group of people. They were clearly quite technically competent. So a lot of what you might call the due diligence and the checks and balances were done right. It wasn't actually until we got the finished product that we realized there was a fundamental problem in what they'd done, and ultimately they were completely unable to fix that problem. So a lot of work, something that never shipped, never saw the light of day. Same experience, I think this happened with some people working on Watson. I see it keeps coming around. I think Johns Hopkins just killed their cancer research work with Watson, but I see again IBM are circling back around now advertising Watson for other applications again. So that's the thing that I think we can all become victim of: the stuff that gives a good demo. I don't know what the rest of you think. I don't know how many of you are actually going into the cloud. Obviously I talked about the world of cloud earlier. One of the things that occurs to me is of course cloud services could be extremely seductive, right? So I think that's one of the things where you can go wading into some of those services and actually get badly bitten. So I don't know, are any of you doing... there's my notes just to spin on the floor. Any of you got any experiences around some of the more sort of operational things when you're trying to transit from one way of working or one environment of working into another in that area?

Everything we do is cloud-based, but picking up maybe a separate point from what you said: I don't think the hype in technology is more often than not clear. It was the case in your case that's not a failure of the technology so much as a failure of expectations, of managing the expectations from the users, the people who require this data and their unrealistic expectations about what they want the technology to achieve. One of the crazy things is that as we see, one of the results of the hype is people come to us wanting to create world-shattering products on day one without going through the necessary and perhaps less exciting part of just doing small steps in front of development. I'm probably the only person here at this event today who's going to recommend boring and predictable as a good way of implementing AI, because I do think that it's not so much the technology, it's managing expectations, coming down to very simple, straightforward, replicable use cases that can deliver as a way of creating innovation.

A couple of reflections on that. I think you were absolutely right: if you can focus on the problem that you're trying to solve and know what success looks like, that's hugely helpful. However, there is sometimes a wrinkle to that, which is if what success looks like is a boss looking great—and I think by the size of the room we've all been there—then you have a real issue in terms of managing expectations. That's one of the things that I've encountered to be extremely difficult, because often the reason that somebody wants a massive project in the way that you describe it is to make themselves look great. I'd be fascinated if anybody had any stories, even from around the room, as to how you let that boss down gently by managing the fact that actually these things are complex. Because actually you've got a second person in the room. I thought the way to manage this sometimes is go slowly, do a pilot, work out whether it works, and then make some decisions and have a little thing to show people. Showing is often so much better than telling in these situations. But there are, in my cynical experience, problems around that precise issue. I'd be fascinated to hear if anybody has a story of how they managed that.

Brilliant. Ian's waiting over there with hopefully a microphone. So just want another story. I heard from a colleague, you know, in a distant division that we don't really have many directs work with, but he's the CTO of this company. He explained to us recently how they do technical evaluation and decision-making around what pieces of technologies they'll use for new products. They'll have the situation where someone will come into the room and be like, 'I want to use this, use that, it's very exciting, we could try this thing.' And they give the instruction to their engineering team: go away, spend a month, build it, but use three different types of technology to build the first prototype. In doing that, you'll learn the pros and the benefits and get under the skin of the project in the prototyping you're thinking about, and then let's reflect in a mature way given those experiences around what are the pieces that we should use. I think that's a really reflective and powerful technique, a very actionable technique for those kinds of questions. And then he said that what happens is the engineers usually come back like three or four weeks later and they're like, 'Yeah, yeah, we're gonna go with boring and predictable.' Has anybody else got any other experiences that they—bearing in mind that this is being recorded and videoed, full disclosure out there—anybody got any other experiences that they might want to share, observations they might want to make?

So I've got actually a question for all of you, and maybe we'll come back to the power saw. So what are the things in the notes that Tasha managed to give you? So I'm going to read this out. This is a quote from her. This happened just a couple of weeks ago when one of our council members—so any of us in the not-for-profit space who have a governance model that involves council members, come on, put your hands up, all of you, there we go, right—wanted us to 'do something with blockchain and fix peer review.' Good thing I wasn't there. Turns out that depositing peer review data with ORCID is that person's particular bugbear. But I think there's an interesting thing there about the levels at which technologies or technology pitches come into an organization. So on the one hand, you've got that very high-level council type thing, or CEOs gone away to a conference and gone to the bar and come back with some great idea. I can see some people nodding and sniggering to themselves, so clearly that's one of those things that has happened. There's the other side to that, which is to say people lower down in the organization have hit on something interesting that they think is potentially useful or exciting, and they're trying to surface that information upwards. So I wonder if any of you out there have got any thoughts on that. Have a think. I'm going to throw to my panelists here with that as a marry question and see what they might have to say to it. Who is going to go first?

I always say if it's going from bottom up, take that person, don't take the idea off that person. Always make sure that that person comes along for the whole journey. For the thing, I'm not going to answer the first times. The simplest ideas, even though you're trying some new technology possibly or you're trying to solve the problem, sometimes the simplest ideas are the easiest to throw out there. Just as an example, Plantwise started just by getting Google Earth, putting some data on it, spinning it, and showing it off.

There's a follow-up to that because I was one of the people that did that piece of work all those years ago. So we did that, that was very good. That unlocked quite a considerable amount of donor funding, which was the purpose of the exercise actually. Then we went away, we built the first version of that product, and we were all very excited when we got that launch. It was our first experience of doing agile, a decade ago now, right? And the then CFO came back and everybody was like, 'This is absolutely brilliant.' And he's sitting right at the front as we showcase this thing for the first time, and he's brown, he's getting deeper and deeper and deeper. At the end of it, I'm like, 'So what do you think?' And he's like, 'Where's the spinny globe?' The thing that he had latched onto was the thing we just got good. It's just the thing to help you wrap your head around it, and he thought it was useless. I mean, it was brilliant for getting money and useless as an actual interface, right? But he latched on to that, and that had become his precious thing. There was a lot of tap dancing done at that presentation to get over that. It's the same as—I don't know if anybody knows the start of Encyclopedia of Life. That was absolutely amazing video of going down like a pipe system through the taxonomic tree of species, and everyone thought that this is how you're going to visualize taxonomy in the future, and so forth. Never came. It's just like another Wikipedia version effectively.

Killing anybody? Maybe this sort of top-down versus bottom-up. Picking up on Ian's point, I do think that one of the challenges with all new technology is that the users, the publishers, are not going to be aware of the capabilities of the technology in themselves. So you can't expect the users to be able to come up with good product ideas without some understanding of the product. So going back to my earlier point of coming up with some simple implementation of the technology, the great benefit of that is it puts sufficient knowledge into the hands of users. And by users, I don't mean technologists at all, I mean end users, who once acquainted with and having some kind of working knowledge of what these things can do, they can then make that ideation process possible. So it's not technology-led, but it's technology-enabled.

I'd like to offer two brief reflections on that. First, I usually say to people it's much better to show rather than tell, unless of course you have the situation you've just illustrated where what you're showing doesn't actually bear what you're finally going to deliver, in which case it's a bit more of a challenge. But showing is far easier than telling. It is incredibly difficult for customers of all sorts, whether they're internal or external, to understand what you mean by something unless you can show them a picture or ideally some sort of interactive walkthrough of what you're talking about. If that doesn't happen, it's super difficult. Second, if that internal person really is passionate about it, you want them to show it and then to frame it in the terms which the management will understand. So telling the story against what you know the management's objectives might be, or what an exit strategy might be, is super important because then it will solve the problems that are front of mind to them. That's a hard thing to do. There is an anecdote I can share about the institution for edging in technology. If you've seen me stand up and give talks previously, you may have seen me talk about the work that we've done with inspection to rebuild an AI database production system. It genuinely does use artificial intelligence, it's also ground truth. We've spent a considerable amount of time completely re-engineering that production process from the ground up, and now we're in the further commercial exploitation phase where we're able to take that work and build a series of data analytics products that are based off that original production back end. So we've completely rethought how we could take an AI database to the next level. Some of you that I know in the rear movers have seen me bang on about that. What I want to talk about there though is that that's been a five-year journey. For the first two and a half years of that journey, the CEO really, really struggled to understand what it was that we were doing. I spent not just myself but a number of times standing up trying different variations of explanations as to what it was that we were doing. Candidly, after he'd finished beating us up for the first year as we were spending lots of money and he couldn't quite understand what we were getting out of it, he reached a point where he was like, 'Well, I understand you're all to a person incredibly excited about what you're doing. I see you jumping up and down going, we really think we've hit something here. Candidly, I don't understand it, so I'm going to trust you.' That was the second year. Then I'm not entirely sure how or when the penny dropped, but one day he suddenly got it. He went literally in the space of one meeting—I was on a phone, I think I was stuck somewhere at the London Excel at a conference trying to dial into another meeting that we were having—when the penny suddenly dropped. From that moment on, he's been one of our biggest champions of what we're doing. The story there is that that's a piece of transformative work that we are doing now for the organization. It's one of the two pillars of work that we are doing going forwards. So I think one of the things I would say is that in all of these things, you see the TED Talks and all the rest of it where these things just happen magically. Sometimes actually you just got to put your nose to the grindstone and grind away, sometimes for quite a long time indeed. Candid truth of it: any number of those meetings could have gone wrong. I probably wouldn't be sitting here because that's the way it goes, right? But there's that hard graft work that often has to happen under the hood as well. I also have a quiet plea. Having worked on both sides of the fence for small and large companies, for publishers and companies that provide to publishers, it would be marvelous if we could get away from dysfunctional RFI and RFP processes. Because if there isn't anything that really accelerates the hype and makes it really difficult to deal with, it is such processes. What you end up with as a publisher or somebody who's looking for a solution is a fantastic laundry list of every single buzzword that has ever surrounded the particular technology that you're looking at, because everybody wants their hat in the ring inside the process. Then as a supplier, you end up trying to manage the expectations around all of those buzzwords, but not being able to have an intelligent discussion with your potential clients about why you think that some of those buzzwords are deeply misleading and problematic. So if there are ways of emerging from that sort of RFI/RFP nightmare, that would be terrific. I would love to hear if anyone else, and particularly given organizations which have quite tight governance structures around such things—and understandably tight governance structures—if there are ways that people have found around that.

Anybody prepared to open limits? I could see people thinking about it. Another anecdote about one major society we were working with. We had several discussions, they liked some solutions. In fact, it was at Frankfurt where we were just about to sign the deal, and they said, 'We've looked at this technology and we think it's so important. Yes, I said that will issue an RFP. They said you're gonna win.' So I think there's two things.

So there was a question that came earlier, and it was asked about innovation groups within an organization. Can't remember who asked this, was it? So actually, hands up who's got a formal innovation group within their organization? One, two, three, four, five. I'd expect you, your group's submitted it. You've got a few. Okay. Is there one of the key points here? I'm struck that there's a key note I saw from a guy called Nick Bilton, who at the time was working for the New York Times. The New York Times had an innovation group—hopefully everybody here will have read it, it's a few years old now, but it's their internal paper on innovation, the struggle to do that at a world-leading newspaper. He came to give a keynote to trade and scholarly publishers, and he said he'd done a piece of work where he'd phoned up a bunch of them and asked, 'What was your R&D budget?' and got the answer back, 'What's an R&D budget?' That does seem to be quite the issue, because one of the things about managing technology hype is that's the function of an R&D or an innovations group: to try and understand these things, either assess them and say yes there might be something here for us, or actually to internally report back and go nothing for us, nothing to see here. I don't know, for those of you that have innovation groups, is that the kind of thing that they are doing? Is not? Anybody else want to add to that? You don't want to add to it but you're noting away. Okay, all right.

I want to say, go on. Yeah. The invisible process which leads to many frustrations, and it's like this beautifully complex handcrafted weird thing. When we're talking about hype, it's often because the language around these technologies that is used, there is a gap between how it is described and what's going on underneath. So I think a role of an innovation group or an innovation lab is to break that down and to enable people to have coherent, mature conversations around what this technology is actually doing, what its strengths and weaknesses actually are. I think that should be one of the core ambitions of what we're trying to do with the innovation group.

That reminds me both of a technique which I think is really important with technology vendors. The story: I co-founded a .com in 1999, no, wait, of the boom, and we raised some money, so of course we had a procession of technology vendors desperate to get access to the venture capital money that we raised. I succeeded in wildly irritating a series of them with a technique which I highly recommend to a number of you, which is basically to say, 'So how does it work? No, no, how does it work? No, how does it work?' Until you get a reasonable answer to that question, don't give up. Because I think to your point, I mean I think you have an important point about translation as well. Just that the role of one of those groups should be about actually translating to people who don't have the vocabulary to be able to engage with those people. But nevertheless, actually getting to that answer, and never trust a vendor who can't tell you how that technology works.

Okay, thank you. I would add to that perhaps the understanding of how something works. I think the role of an innovation unit is wonderful when it communicates how the technology works. I think the challenges that sometimes happen as a result of the hype: many publishing organizations say, 'We're going to resist the hype, we'll make sure that we evaluate this technology very thoroughly. You'll find a unit in the corner who are experts, and they've all got PhDs, and they can all answer their questions, and we'll leave it to them.' Trouble is, the experts in the corner then can't communicate their results back to the rest of the organization. So the net result is no understanding of what this technology does, and that's not where we want to be either.

Absolutely. Got a question over there. Can we get a mic over there just so we can thank you? There's other questions or comments to what Nick was saying.

Not only should answer how does it work, but why should it work for me and why should I have it? I think that's another part of the same coin. There's a lot that relate to that.

I completely agree. It's not realizing my objectives and my goals. It might work but not in this context.

I think I would add to that further. I mean, not only that, but how can I measure that it's been successful in my organization? In other words, what does success look like? And I want to be able to hold you to account for exactly what those results are going to be. And if you could put that in a contract.

Thanks so much. Can I go back to innovation groups just for a second? I can imagine what one of those looks like at a very large business, so you're going to medium-sized business. And I know what innovation looks like when it's an individual within a business coming up with an idea, and we've talked about that already. What does an innovation group look like at a business of 25 people?

I think it looks like projects. So I think it looks like working on an idea with a group of people of whatever is an appropriate size to investigate something. So having time carved out to explore and play, rather than it necessarily being anything more formal than that. So that can—I've seen that happen. So it's about making the conscious decision at a management level to give people time to play and to explore.

What's the message?

So there are two messages. It depends on which witch you want to choose. One is fear and one is hope, right? So there is fear of being left behind and hope of doing something fabulous which will give you growth. It could be both, but generally those are two pretty good emotions to appeal to. I think one of the other things that might be useful, again coming back to the work that Phil and I were exposed to when CABI together, was she had a CEO came in, had a long-standing but essentially moribund organization—let's be blunt, a moribund organization—and he came in. There are a few things that he did, but the one thing that actually really sticks with me and still sticks with me to this day because I think he absolutely got this right, was he really pushed through the concept of the big hairy audacious goal, the BHAG. Which sounds like one of those hideous horrible business Harvard Business Review things, but actually if you can get that right, you can get an organization to coalesce around a conceptual idea of what it is they're all in the business of doing. Plantwise directly sprung out of that big hairy audacious goal. The audacious goal was: how do we turn ourselves into a globally relevant organization for the community, the developing world, that we are supposed to be solving the specific things of? Plantwise popped out of that. But there was a piece of work to get a very disparate, very siloed, poorly communicating organization with a huge amount of talent to come back together and focus on something. That was a two and a half year, three-year project, and it was bloody. Let's not beat around the bush: a bunch of people didn't survive that process. But the organization that came out of that fundamentally was in a much, much better place. I think probably to this stage, you could see from some of the stuff that I've brought on, it's fundamentally better structured to think innovatively about how it delivers on its mission. So the BHAG is one of those things that I think organizations looking for direction could really usefully pay attention to.

No, go ahead. Go, getting a microphone. So to build on that a little bit, I mean, I would say this: I'm a strategy consultant, we know what a strategy is. Ultimately, the articulation of a strategy gives you a yardstick against which to judge a great many things, including the technology hype, because it gives you a very clear way of saying that's relevant, that's not, let's investigate that.

Actually, just connecting a couple of points you made and actually your story. Interesting. I did set up an innovation group and eventually abandoned it. The reason I abandoned it is if we made it from the people who had had the great ideas in the past, the pressure to come up with a good idea is very difficult. So come back to your point, I think you've got 25 people in your innovation group. The issue is how do you bring that together. And your point about your chief exec who didn't really understand exactly what you were doing but was willing to trust, it occurs to me that you maybe ought to change the third word in the title from 'managing' to 'harnessing'. Because if he'd been managing technology hype, that project wouldn't have gone anywhere. But if it was harnessing the technology hype in conjunction with good business goals, you're actually moving the thing forward. So I think it's more harnessing than managing in my view. And it's managing where do you get those innovative ideas from. I think sometimes having an innovation group actually stifles the real innovation in the rest of the business.

I totally agree with that. And since David's left CABI and whatever, we moved from an innovation group actually to a solutions team, outside of IT, within the business, of being able to take everyone's idea and try and help them to lift it up to the next level effectively. I think your knowledge is if you want to cook a meal, you don't have to build a cooker. You need appropriate understanding, and it's business solutions you're looking for rather than understanding every bit of the technology that you're using.

Okay, well I know that we are sitting between lunch and I don't want to get caught in the stampede. So all I want to say is thanks very much to our panelists. If you'd like to join me in giving them a round of applause. I really enjoyed the talks this morning. I believe lunch is outside. Is that right, Joe? Out to you, sir. Lunch, let's go.