DC Fintech Week 2024: The Lightshow: GenAI, and The Difference One Year Makes

Ladies and gentlemen, it's as Chris mentioned. I have homework, or had homework if you will, to put together not only this presentation but a demo for you. And not to rest there, but then Priscilla is going to come up and make sure that I don't get to just walk off stage to thunderous applause because we did a great demo, where she's going to really grill me with some great questions. I hope you'll find them interesting. So let's get started. I think we all know how innovation really can transform industries. There's been major technological innovations over the past 20, 30 years, and every time they come in, it's an amazing inflection point where now we're almost all set equal again. There's a brand new thing that we're all trying to scramble and learn about and how we can apply it within our industry. For me, the last big one might have been mobile applications, or the search engines for the internet, or cloud computing where I don't even have to buy computers, I can get access to technology. So major innovation that can transform industries. And I think a lot of us are now confronted with this new wave of AI capability that we're calling generative AI. And quite frankly, last year we were all scrambling to learn about generative AI and we had lots and lots of questions from our customers. Just basics: what do you mean by generative AI? Is this something that's secure? I'm sharing information, sometimes even personal information, with something that sounds like a human. Is that going to be secure? I heard about this new capability, I need to bring into my organization. Do I need to become a prompt engineer? What does that even mean? How do I train people? How do I evaluate if someone's a good prompt engineer? How do I choose from all these models that are out there? These were the kind of things we were getting from customers. Everyone was starting their journey to understand how this transformative technology might be something they can apply to their business to take a leap forward. But now we've started to get some answers, we've started to get some clarity, we're starting to get information about what the regulatory environment might start to look like, questions that we should be asking, and answers that other people have discovered. And now we know that we can get started. I heard on one of the earlier panels the key element here is to engage with the technology, understand what it can do for your business, understand what it can do for your organization, and start to see where you can apply it. We're seeing customers go beyond proof of concepts, they're starting to go to production applications, they're starting to customize these models so that they're being purpose-built for specific use cases. They're doing things that they can deploy them and then thinking about what does the throughput look like, how do I make sure that I can scale this solution outside of the lab and into thousands of users in my organization, and tens and hundreds of thousands of users if I'm going to be having that capability for use with the public or my end users, my customers. I now know when I want to select the model for which purpose, maybe for speed, maybe for accuracy, maybe for thoroughness, maybe for broad capability, maybe for targeted information and deep expertise. I now though need to manage risk as the regulatory environment continues to evolve. There are over 500 pending pieces of AI legislation across the United States at the state level, and we're seeing governments around the world grapple with what it means to think about regulating this technology. And what we found is it's kind of hard to predict where those are going to land, and so you need tools and capabilities to react to that. You have a foundation model or an AI capability and you say I want to protect against this threat, or I want to make sure that we accommodate and react to and respond to this piece of pending regulation. So you need guardrails and tools and capabilities to make sure that as you build applications using generative AI technology, that you have the capability to be able to respond to pending regulations. Then how do you measure success, manage this, how do you do the DevOps of generative AI? When do you decide when it's time to change a model, update a model, update its weights, update the tuning that you've done? How do you build the operational monitoring around something where you now have made it integrated into an application? And how do I move fast? A lot of you are probably responding to boards who are saying we got to implement and deploy generative AI, why? Because we need to, because everyone tells us we have to. So how do you measure that actual ROI, that success? Do you have ways to do that? And we're seeing customers actually come up with yes, look at the time savings, the productivity benefits, and the new capabilities we're able to offer to our end customers. So in 2024, we're seeing this go to production, go to implementation. Early days for sure, but in many ways extremely impressive. The range and creativity of different kinds of applications I'm seeing from customers is astounding, it's really amazing. But let's go beyond that a little bit. Let's talk about what some of these significant business values actually can be. There are new experiences you can create for your customers, there are ways you can make your own employees radically more productive. An AI assistant for your employee that can allow them to do things they could never do before, or do routine things much more quickly, take vast amounts of data and information you may have available inside your organization in different formats, gather and extract insights from that. And then last, this area is the most amazing one for me, is how can you be creative, create new experiences, create new things, new content, and use this technology to provide that for your organization.

After you log into the portal, you'll see applications that you can work on. Let's look at the quotation. The screen shows all files that you've uploaded previously for this application. The system automatically tries to identify the files which contain the applicant information and list of properties that they want to ensure. You can override it by selecting from the corresponding drop-down menu. Next, the system shows a side-by-side comparison of the extracted applicant details and where it comes from in the original PDF file. You can then validate the fields and correct the entries if needed. In the next step, the system extracts and displays the list of properties to ensure, including the address, postcode, and some to ensure. When you click on a property, the screen will highlight where the extracted data comes from. You can then verify the field and modify them if you need to change anything. In this step, you can see the details of each of the properties. Let's take a look at the first property which has a warning sign. The system automatically enriches and summarizes the property information from different data sources. In this example, it finds a potential roof leakage problem by using AI to analyze property photos. Another example shows how it uses AI to summarize the customer financial situation and claim history. In the final step, the page shows the premium and the coverage. You can review the clauses and make changes to the coverage and excess amount. Here you can see that a decrease of the maximum amount payable for the locks and keys clause reduces the premium.

One of the things I love about that demo is that it is multimodal. One of the things that's fascinating about it, if you could see through what it was doing, is it took printed forms with handwriting and extracted the handwriting data and got it into your system for you. It took PDF files and interpreted them and changed them into different formats. You'll notice all of the data that was shown was kind of in these items and then these tab bullets. And if you looked closely, it would tell you well this came from this file and this came from this system and this came from that system. And what it's doing is it's unifying the content for you so you can work with it in a similar way. And then of course we do our multimodal thing where we're taking a look at imagery, potentially even video, to identify what's going on in that situation and understand how it affects what we're trying to pull together from a data perspective. So this is a system that's integrated with generative AI. It's not a generative AI chatbot, it's rather a business process and business flow that's saying how do I use AI to enrich that flow through every step. And if you've been paying attention, even today one of our partners, Anthropic, announced a new capability for generative AI that allows it to interpret what it sees on screen and allow you to instruct it how to edit and move the cursor around and interact with your application on screen. So we're really starting to accelerate this kind of capability. It's really amazing. In the demo, the person's going in and typing and changing changes. In the future, you can essentially say move to that field, edit this, and change it to that, and it'll follow your instructions. So super amazing what's possible there. But what's really key about this is that amazing amount of data that you might have and putting it to work using generative AI to transform data from one format to another so you can have a unifying experience for a particular business process and application. So your data really becomes now active data. All that data you may be sitting on, you've been gathering it in different formats, you've been trying to find ways to unify it. Generative AI makes that so much easier so that data really becomes something now you can put to work. One of the questions we continue to get about generative AI is around its security. How do we really make sure that we've got the governance, how do we make sure that we've got the controls, how are we thinking about the output of generative AI from a legal perspective, how are we making sure that generative AI respects the privacy rules and regulations that we want to comply with, how do we manage the risk around these things? And these are great questions. And the good news is people like AWS are thinking about this from the start, whether it's security baked into how we make it available or giving you a guardrail so you can mitigate and adapt to the questions you're getting. We've really been consciously thinking about embedding that and putting that into practice. But really it comes down to innovating responsibly and understanding yes, this is a great tool, great power, great capability, but how can you continue to use it in a way that is compliant with what you need to do from a responsibility, legal, and regulatory perspective. At AWS, we think of AI in multiple dimensions from a responsibility perspective, all the way from fairness to ensuring veracity and robustness of the output of the model. Some of these may overlap between explainability and transparency, but also there are key elements here around your ability to control what's put out by a model, to govern the data that goes in and what comes out, and then how do you maintain privacy and security and be really clear and honest and open and transparent about the safety measures that you're taking to ensure that the data that comes out of these things is used responsibly in a responsible manner. These are evolving areas. All of them sometimes have a touch on either regulation or a touch on how you're going to define your policies around using this. These are questions we get from customers and it's super cool to see where they are navigating what their internal policies will be on their journey to implementing this technology for their benefit, for their applications. So one of the things we've talked about and we've learned kind of at a high level with customers is most importantly to engage your stakeholders, put people first. Generative AI has the possibility of interacting and actually in many ways feeling like yet another person in your organization. So it's super important as you expose your employees, your potential customers and users to interacting with the output of a generative AI model that you engage them in that process, have them be a stakeholder, use focus groups for testing, have an approach that allows you to understand who's going to interact with these models and understand the risks about the use cases that you're working on. Have an understanding of which use cases you want to have and what risks are associated with those use cases. We heard on one of the earlier panels the difference between generative AI for a cafeteria menu and potentially applying for a loan. Iterate as much as you can, test, test, test. One of the things we see with customers is those who have a mature test and evaluation framework or an idea how they're going to evaluate the performance of a model really get further along in their ability to deploy models into production when they have a robust test and evaluation framework. What does the future hold? Well, I mentioned the Anthropic announcement from today and we're really seeing some amazing things with the ability to audit and automate the output of generative AI in a human-supervised way. We're seeing multimodal models that can adapt and work with voice, with text, with imagery, with video. It's amazing what you can do when you start to really get that fusion of data that we've all been hoping for. Understanding that you're going to probably use a different set of models, multiple models for different use cases, and understand that you're probably going to want to make sure that you're evolving and implementing policies and standards to really make sure you can take advantage. So select the right use cases, empower your teams to innovate, and get started on those top use cases today. So with that, I think yes, I think we're ready, Priscilla, if you want to come on up and interrogate me and give me some good questions. I'm looking forward to it. Thank you.

I think one of the first things we saw was the immediate reaction of many organizations was, oh my gosh, we have to ban use. We don't know. A lot of these first generative AI things are out on the internet. We're uncomfortable that our employees were putting information out there to these models outside of our control. So the early reaction was shut it all down. And then what we started to see is organizations that were thinking about what they could possibly do brought generative AI inside the organization and allowed their organization to get experience. And here's what I found out: when you made it broadly acceptable for use internally, the use cases would come organically from places you wouldn't expect. When we made generative AI approved for use inside AWS, we got use cases coming out of legal, out of marketing, out of engineering, places that you as an IT person would think, we need to look at an IT problem first, but we're actually getting really interesting use cases bubbling up from the org. And so that's where I really encourage people: make it broadly approved for use internally and allow people to experiment and propose use.

That's a good one. We found that basically what it was, sometimes early on we're at a phase where we're like, okay, how do we do something that maybe is low risk, has potential for high productivity, has a small enough user base that we can experiment and learn without huge amounts of people being exposed to something we may not fully understand yet. And so the first use cases were internally focused, productivity based, small user counts. How do we learn from that use case? That was kind of where you start, how you prioritize at first.

We found that some of these models are so big and they work well when they have a lot of data that cloud's a natural place for that because you can scale both compute and storage to accommodate the appetite potentially of a large model. So that was kind of the first place. But we also found it's interesting: organizations with a huge amount of data often, as the first use case, were saying I need generative AI to help me distill this data, to help me not search it so much but get insights from this big pile of data in response to more unscripted questions. So in the past where we might have had very structured data warehouses and data analytics teams and data flows to get data into a format so we can get insight, we found companies and organizations with large amounts of data, usually unstructured in multiple formats, saying can generative AI help us make sense of it? So early use cases were really kind of search and what we call retrievable augmented generation, or a generative AI that cites sources. Early on, organizations that didn't have a lot of data wanted to use generative AI for the creative aspect. They're saying we need a sourdough starter, like ask the generative AI model to generate a starter kit or a starter template or something we could riff on. And so those organizations tended to use the more creative aspects, whereas ones with large amounts of data used more analytic aspects.

It's a great question. I can tell you our position and what we recommend to clients and customers. First of all, we have a high bar of saying you customers own their data. So the data environment starts out with data you already own, and then as you use generative AI to transform that data or to create new data, that data continues to be owned by you because you've instructed the model to do a thing with your data and create new data outputs. And so the key message here is you continue to own your data and be responsible for the data that is coming out of generative AI because you've been instructing it and putting guardrails and constraints potentially on generative AI as to what it can and cannot say. It's not a lot different from the constraints or review you might do with your own internal people who create data in an organization. You continue to own it, you have to review it, you have to understand its ramifications whether or not you're going to publish it. You apply those same well-known constraints and mechanisms and processes to data that's created or generated by generative AI.

Yes, anything.

The first thing, it's your model, your cloud.

So the first thing is just like today, when you provision your environment on AWS, that's in the data plane. We have no access to that. That is your data. We have no access to it. It's encrypted with your keys. We don't have access to your keys. So your data today as you put it on the cloud is safe and secure because it's controlled by you. You have the ability to put it there, take it away, encrypt it the way you want. We can't see it. When you on AWS work with a model, what we do is give you a copy of the model in your environment. At that point, we have no access to the model anymore. So you will tune it, augment it, change it, fine-tune it with your data. We don't see that because we've given you a copy of the model. It runs in your environment. It's yours to play with and do what you need to do, what you want with. So it's not our model at that point. We've given you a copy to work with. And so we feel like that is a way to make sure that we're continuing to maintain the integrity and privacy and security of the data in your environment. The model becomes part of your environment that you maintain control and have access to just like anything else. We're built on open source technologies, so we feel like if you want to pick up and move to another provider that provides similar open source technologies, we feel like we're building something in a way that's compatible with that. And so we feel like if you need to do that, you can. We just want to continue to provide the best experience and services so you don't feel like you need to.

It's a great question. Even in my own organization, I have a project management team and they help run some of our operational capabilities. And one day I just started to show them some of the things I could do with generative AI. And it was a simple thing where I took an Excel spreadsheet, I dumped that into generative AI, and I asked questions about the data in the spreadsheet and got formatted answers in a way I could present in a document. And my program management team, their heads just exploded. They're like, oh my gosh, do you know how much time it takes us to convert from one format to another, to gather data, collate it, and make it presentable? And I said, well, here's some tools, go to it. They come back to me every week with look what I did, look what I did, look what I did. And so this is that idea that it's impacting multiple kinds of jobs, that it's a creativity and productivity tool. Sometimes I worry that it's bounded by what we can imagine. And so one of the interesting challenges I found is if you're of an age like me, you're used to searching the internet with like five words, what's the minimum amount of keywords I can get to get the right result back. And what's fascinating is with these generative AI models, it's actually the reverse. You want to provide a lot of context about what you're trying to do so you get a better result. And so we have to unlearn some behaviors or technical behaviors we've done in the past. And so in this case, I think some people who maybe come in with a fresh approach are going to have more advantage there. So it's funny when you say skills gap, it's not only people who don't have skills, but those who have skills that we need to kind of maybe unlearn or relearn or modify and adapt.

I do. I think the people who are able to really get the most out of the productivity benefit of using these models, you're going to see expertise in doing that. How do I get the most out of a model? I think you're going to see jobs in the model training and tuning space a lot too. What we're seeing with customers, I talked about use cases, we're actually starting to see customers choose different models for different use cases for whatever reason, whether it's speed and accuracy, cost, domain knowledge and expertise. Some of the really large models are starting to become very generic in their answering because they're trying to get ahead of potential regulation too, and so they're becoming very generic in their answers. And so we're seeing customers say I want specific models, which means I'm going to need domain experts to train, supervise, oversee the model itself. And so that's going to be a new role as well.

I think this gets to the bright line for me is if today you have a process by which you use legal review or you have an internal process to review content, and you're thinking about using generative AI to create content in your organization, you must use those same procedures with that output. I have some customers who are like, well, I actually, I'm like bring legal in, bring legal in. They're like we bring in legal in early. And I said, well, today when you produce content and you're going to publish it on your website, you likely have a process. It's got to go through legal review, plain language review, all this stuff. You cannot, in my opinion, say oh, well it's AI, we don't have to do that. So for me, the bright line is if today you have constraint, you have ways to protect your liability, ways to protect your security, ways to oversee and understand the quality of the data your organization produces, you have to use those same processes with generative AI. When you don't, you're really courting trouble.

My pleasure. Thank you so much.