CEOInterviews.AI
Start App
Get an email when Ankur Moitra gives a new interview
One email per new interview. Unsubscribe in one click.
Ankur Moitra
Professor of Mathematics, MIT

Lecture 2: Independence and Conditioning

📅 Dec 17, 2025 MIT OpenCourseWare 70 MIN 7876 VIEWS 122 SEGMENTS · 2 SPEAKERS
MIT 18.200 Principles of Discrete Applied Mathematics, Spring 2024 Instructor: Ankur Moitra View the complete course: https://ocw.mit.edu/courses/18-200-pr... YouTube Playlist:    • MIT 18.200 Principles of Discrete Applied ...   We explain what it means for two events to be independent, and what it means to condition one event on another. We cover the law of Total Probability and Bayes’ rule. We then define random variables and expectations, and explain linearity of expectation. License: Creative Commons BY-NC-SA More information at https://ocw.mit.edu/terms More courses at https://ocw.mit...

Questions asked in this interview

1
  1. 25:12So what are the nodes, the bottom?
Ankur Moitra 0:13 ↗
All right, everybody. Welcome back to class. So today we're going to pick up where we left off. So remember, last time we talked about the pigeonhole principle. I showed you a whole bunch of applications of it. And then we started this longer topic that's going to take us a couple of lectures, which is probability. So just a quick recap of where we left off last time.
So last time I started to introduce the basic terminology that we need for probability. We had these key definitions, like we had the notion of a sample space, which is really just telling you what are all the possible outcomes of your experiment. You could think about something very small and finite, like I roll a die and I get a number between 1 and 6. Or you could even think about a sample space as being an infinite space.
For purposes of our class, we're only going to consider discrete sample spaces, but you can even consider sample spaces like what happens if I take a coin and I flip it as many times until I get my first heads? That's something which has an infinite sample space. And what's important is that to the sample space we associate what's called a probability measure P. So all this does is it takes in some particular outcome from the range of all possible outcomes. It better assign a non-negative probability to that.
And additionally, your probability measure had better be normalized so that the sum of p of x over your entire sample space had better be normalized so that it's equal to 1, so that these probabilities make sense. In fact, there are a lot of distributions that we're going to see over and over again. One of the simplest and most ubiquitous distributions is called the uniform distribution. So here, we have some finite sample space S. And then we just look at the probability measure that assigns the same fixed probability to every possible outcome. So in particular, in order for it to be normalized, each of those probabilities had better be 1 over the size of S.
So this is where we left off last time. We also defined this notion of what an event was. Sometimes we're not interested in one particular outcome, but really a collection of outcomes. Something like when I take a die and I roll it and I get an even number, that's not just any one atomic outcome, but it's really three outcomes for 2, 4, and 6. And we talk about the probability of an event as really just being the sum of the probability measure over all of the outcomes that land in the subset T.
So this is where we left off last time. And today, what we're really working towards is we're going to talk about something that's very important, but a little bit counterintuitive. We're going to talk about independence and conditioning. So what's important to build up to this is really the idea that we cannot just consider the probabilities of individual events, but we can combine them in interesting ways. So for example, if I give you two events, T1 and T2, I can think about the union of T1 and T2.
What this means is I'm really just treating this as the set of outcomes I'm interested in. This is another set of possible outcomes I'm interested in. And I'm forming a new event that takes everything that happens in either one of these or maybe both. So in particular, in words, what does this new event that I've constructed mean? It's the event that at least one of these events, T1 or T2, happened.
So that's one way to combine events. I could also think about combining events by, for example, taking T1 intersect T2. And then what am I going to get? I'm going to get a new event that's asking me whether both T1 and T2 happened. And so this now leads to a very interesting concept, which is the notion of conditional probability. And I promise that this will sound very simple. But in the examples we'll see, some of them will become pretty counterintuitive. In fact, when you look, conditional probability is really in the world all around you in very important ways. We'll talk about some real world examples.
So first, let me tell you what this means. Let me just give you the definition. We can also combine two events, A and B, in this interesting way, where we want to assert for a fact that event B really did happen. And we want to look at if I tell you that event B happened, what's the new probability that this other event A happened. So you can think about this in a few different ways. So in words, it's the probability of event A given or conditioned on that event B happens.
See, one way to think about this is really in terms of pictures. So we could start off with our full sample space S. That's all of the different possibilities. We could take one event A right here. And what's going on is that our probability measure really describes a way of throwing a dart at this board. Now, this dart always lands inside this box S, because that's the range of all possible outcomes. And then the probability of the event A is just the probability that that dart lands inside this oval that I've drawn right here.
So now when I have a pair of events and I have A and B right here, well, what I can do is imagine I do this experiment where I throw my dart at the board. And we already know it has to land inside the set S. But then what if I tell you that it's landed in this set B? And I promise you that event B happens. What I want to know is, given that promise, what's the new probability that it actually lands inside the set A? So the only place where it could then land and event A could happen is right here in the intersection.
And that's exactly how we define the conditional probability. It's literally just the probability of the intersection of the two events divided by the probability that I'm conditioning on. And this part is intuitive. Imagine that I have the same chance of hitting every single point inside this rectangle. Then really, what's going on is that when I tell you that event B has happened, the probability of B is how I'm renormalizing my space of possible outcomes, because I know that now my dart is uniform over the set B.
And what's important is not the amount of space that A takes up inside S, but the amount of space that A intersect B takes up inside B. So does this make sense? Are there any questions about the definition of conditional probability? Good? All right. So let's develop a few different ways to think about conditional probability. Let me actually give one other simple example, which is related to it, which will have its own quirks.
The other crucial notion that I want to introduce today is the notion of independence. So we talked about two events, A and B. And there's this very important concept, which we're going to use all the time, which is called independence. So we say that two events A and B are independent if the probability of A intersect B is equal to the probability of A times the probability of B.
So this is, in some sense, the simplest way that the probability of intersecting two events could work is that whether A happens, you think of it as it contributes this P of A. And whether B happens contributes this P of B. But in fact, just as an exercise, using our definition for conditional probability, we can see that independence is really equivalent to a much simpler thing, which is that probability of A given B is equal to the probability of A.
So this comes just from dividing through by the probability of B and using my definition for conditional independence. So these are two completely equivalent conditions. You can check either one. And that's what it means for two things to be independent. So what's going on is that in my dartboard example in pictures, I might have my entire rectangle represents S. I might have this left half represents A. And maybe this bottom half represents B. And so what's going on is if I look at the probability of A, that's how much area this rectangle for A takes up out of the entire rectangle, but then when I condition on B, it doesn't change what that fraction is.
So this is one way to think about what independence is. And I want to caution you that even though it's a very simple definition, it's actually very counterintuitive in some ways. There are easy pairs of events that are natural to think about why they're independent. And then there are some events that are independent that are very counterintuitive. So let's do the simple one first. Let's do some simple examples to get some intuition for independence. So imagine that we throw a dice twice.
And this is a fair die, so it's the uniform distribution over all six possibilities. And when I look at the pair of throws, my sample space now has 36 possibilities. For every possibility for the first throw x1, every possibility for the second throw x2. So I look at all possibilities of x1 x2 that are in this set 1, 2, up to 6. So this is my sample space, and everything is uniform over the sample space.
All of the 36 possibilities for the choice of some number between 1 and 6, and some number between 1 and 6, are all equally likely. And now I can define some events. Let's look at events A, which is that x1 is odd. And let's look at the event that B, that x2 is at most 3. So probably you all have very good intuition for this. So who thinks these two events A and B are independent? OK. Who thinks they're not? Who is still tired? OK. All right. Good honesty. All right.
So these two are independent. Now, one way to check independence is literally just to follow the definition. So when in doubt, just compute the thing. But we're going to see part of the intuition for why they're independent as we'll see in the computation itself. So what does it mean to be independent? Let's check. Let's look at the probability of A and B. So in our sample space, there are 36 possibilities. How many possibilities meet the condition that both event A and event B hold?
Not a hard question. Yeah? A quarter of them. Yeah. So let me just say 9 out of 36. Why is it 9? Because I have three possibilities for x1 is odd: 1, 3, 5. I have three possibilities for x2 is at most 3: 1, 2, 3. And every choice of one of the possibilities for A and every choice for one of the possibilities of B defines a possible outcome that satisfies both A and B.
So they're independent because they have this kind of product structure that whatever happens for A, I can choose something there, and then I can define something that happens for B. So literally, if you think about it, it really represents this picture, if you think about it the right way. So in any case, we can check that this really does equal the probability of A times the probability of B. Because what's the probability of A? It's just 1/2. 1/2 of the rolls are odd. What's the probability of B? It's 1/2 because 1/2 of them are 3 or less.
So the way to think about it is that independence is sometimes intuitive. Because what's going on here is I could literally think about rolling two separate die. I roll one die here, and I care about what the roll is. And then you go to another room in building two and you roll another die. And they're independent, because we literally are not talking to each other. These events are just happening in different spaces.
But independence is a bit more counterintuitive than that, because there are other ways that events can be independent that satisfy our definition, that aren't so easy to check. Let me also mention that a lot of times we use the shorthand, which just means that A and B are independent as events. All right. So let's go a little further with this example so that I can convince you that independence is subtle. So let's define a new event C, which is going to be the event x1 minus x2 is equal to 3.
Now, this is a bit stranger of an event, because it involves both x1 and x2. Now, what are the possible outcomes in the sample space where event C happens? Can anyone list them for me?
Student 14:56 ↗
[Inaudible]
Ankur Moitra 14:59 ↗
Perfect. And those are the only possibilities. So another way to define this event is really as the subset of the sample space, which we're interested in, which is the possibility 4, 1. That's the possibility 5, 2. And lastly, we have the possibility 6, 3. That's the only subset of the sample space that I'm interested in. Those are the only things that can happen where this event C is on, is true.
And we can look at the same type of thing. Let's look at our definition for independence. Let's look at the probability that A and C both happen. So what was A? x1 is odd. And there's only one possibility on my list where x1 is odd and C happens. So this means the probability of my intersection is 1 over 36. Unfortunately, this is not equal to the probability of A times the probability of C. We know the probability of A is 1/2, and we know the probability of C is 3 over 36, just based on our description of the event.
So these two events A and C are not independent. A is not independent from C. And maybe this is intuitive too, because these two events depend on the same things. They both depend on what happens for roll one. But what I claim is that there are slight changes to event C that actually make these things be independent. Let's define a fourth and final event. Let's look at this event D. That's x1 minus x2 is equal to 4 instead of 3.
Now we have only two possibilities. We have the possibility 5, 1, and we have the possibility 6, 2. And we can just check our definition for independence. So the probability of A and D. Well, there's only one thing that survives. So my probability of the intersection is still 1 over 36. And you can check that this really is equal to the probability of A, which is 1/2, times the probability of D, which is 2 over 36.
So this is an example which would be a little bit hard to guess unless you literally did the calculation, because it doesn't look like they're independent, but they still are. So are there any questions about independence? Does this make sense? OK. All right. So let's go a little further. I want to tell you about some more powerful tools in probability that play very nicely with conditioning and independence.
Let me tell you about this basic fact that we're going to use all over the place. And then I'll give you a way to visualize it. This fact is called the law of total probability. Even though we already defined whatever we need, just in terms of the sample space and the probability measure, there are a lot of tricks that are going to happen where there are clever ways to compute probabilities that don't involve listing the entire sample space and just adding things up in a boring way.
So a lot of these tools are really a way to organize and think about the sample space to try and make probability more intuitive. And the basic version of the law of total probability is that if I have an event A and I care about its probability, we already have a definition for how to compute that probability just by going over the entire sample space. But what I claim is that we can break it up into two different terms, one where A and B both happen. And then I can add in the probability that A happens and B does not happen.
So this is just if I'm interested in the probability that A happens, well either A happens and B happens or A happens and B doesn't happen. These together cover the space of all things that could happen so that A still occurs, and they don't overlap at all. So in particular, the proof of this is there really is no proof. It's literally just we take the definition of what the probability of A is. It's the sum over all x in my set A of p of x.
And all I'm doing is I'm breaking the sum up into two pieces, it's the sum over all x in A and B of p of x. That's literally this right here plus the sum over all x in A and not B of probability of x. And so pictorially, we can go back to our favorite diagram. We have this event A that we care about. We have this other event B. And then we can break up this event A into two pieces, one where A happens and B does not, and one where A happens and B does. And together, they cover the space.
So, in fact, there's an even fancier version of the law of total probability, because no one says you have to do this with one other event. In fact, you can do this with a ton of other events. So let's talk about the most general version of the law of total probability. We can be interested in the probability of some event A, and we can write this as the sum over a collection of other events of the probability of A and Bi. But now we need some conditions on the Bis.
We need that all of the Bis are disjoint. So Bi intersect Bj is the empty set for i not equal to j. They're pairwise disjoint. And we also need that together, they cover the entire space. The union of all of these Bis is my sample space. So in particular for this most general definition for the law of total probability, really, the way that I recover that special case we already talked about is I consider two events, B1 and B2, and B2 is the complement of B1.
And that way, I just get two terms in the sum. They're definitely disjoint. B1 and its complement don't intersect. And together, they cover the space. But instead of breaking up my entire sample space into only two possibilities, I can think about breaking it up into three, four, five, how many ever is convenient. So any questions about the law of total probability? Good? All right.
So let's go back to visualizing these things. I want to try to make these things as intuitive as possible. So in fact, there's a clever way to think about them using a diagram, which I find very helpful. So we can think about our random process as really taking some sort of path in a tree. Maybe we start off our random process, and then we have a whole bunch of possibilities, like B1, B2, B3. So what I'm deciding here is really the answer to the question, which of the Bis happens?
Because in the setup of the law of total probability, all these Bis are pairwise disjoint, and they cover the space. So when I start this process, before I actually tell you what the outcome is, where the dart lands, I'm first going to tell you which region the dart lands in. Does it land in B1? Does it land in B2? Does it land in B3? The assumption that these things are pairwise disjoint just means that one and only one of these things happens.
But I have a different probability of traversing these edges depending on what the probability of those events are. And then finally, I can look at conditioned on B1 happening whether the event A actually happens. So I can continue this process. And then what I can ask is, how does, knowing that Bi happens, how does that change the probability of A? So this is a nice way to visualize it.
Because what's going on in here is you can write it out instead using conditional probability. Another equivalent way to do this is it's the sum over i of the probability of A conditioned on Bi times the probability of Bi. Literally nothing is happening here. I'm just using the definition of what conditional probability is. And then you can see that algebraically, what's happening is first you're telling me which Bi happened. That contributes this term right here.
And then once you tell me that Bi happens, I have to also take into account the remaining conditional probability. And when I add up over all the possibilities, that gives me the total probability of the event. So these are very basic algebraic manipulations, but they're powerful. So any questions? All makes perfect sense?
Student 25:12 ↗
So what are the nodes, the bottom?
Ankur Moitra 25:15 ↗
Right. So this would be these are all of the possible outcomes in the sample space. So what I'm doing is instead of just telling you first where the dart lands, like which x happens, first I'm going to tell you whether B1 happens, B2 happens, or B3 happens. Once I tell you that B1 happens, I know my dart lands in that space. I look at all the x's that belong to B1, and maybe I just shade in the x's that belong to that subset that are also in my set A.
So for example, the fraction of nodes here that are colored is exactly the probability of A conditioned on B1. So you can see that I have this probability of taking B1, and then I have a conditional probability. Or I have the probability of taking this branch and then another conditional probability. This is pictorially what the law of probability is telling you. You can think about it many different ways. This is tricky, so don't be embarrassed to ask questions. That's why I'm here. Any other questions?
Everyone thinks this is way too easy? Way too hard? Just right? OK, I'll take it. All right. Let's see. All right, let's check our understanding, because some of these things are a little bit counterintuitive. How many people have heard of the Monty Hall problem?
OK, how many people feel totally comfortable with Monty Hall problem and think I should skip it or how many people want to hear it? Who wants me to skip it? OK, that's a very small number, so going to do it. So you've all seen the Monty Hall problem before. What I claim is that a lot of these ways of thinking about things in terms of the law of total probability, in terms of this visual I gave you here, they give you a nice intuitive way to think about what's going on.
So I'll give you maybe an explanation you haven't seen before. So remember in the Monty Hall problem, there's a game show and there's three doors. And behind one of the doors is a nice prize. Maybe there's a car. But you don't know which door has the prize. So the way that the game works is that first, you pick a door. Now, whatever door you pick, if I pick this door, there's definitely another door that doesn't have the prize.
If I pick this door that has the prize, there's still another door that doesn't have the prize. So either way, the host can always, no matter what you pick for your first door, reveal to you another door that you didn't select which does not have the prize, and show you that you shouldn't pick that door. And the basic question is, do you switch? So this is a very famous example. I think the first time you see it, it is a bit hard to wrap your head around.
But it's been done in virtually every class that covers probability so often that maybe it's become a little bit more intuitive. As you all know, the answer is that you switch, that then you'll win this game with 2/3 probability. Now, you can figure that out by computing conditional probabilities. You have to be very careful about the definitions of your events. But I think a very intuitive way to think about it is in terms of this tree that I drew before, because we can think about all the things that can happen in the game.
So in the beginning, when you choose a door, remember that the prize is put behind a door at random. So whatever door you choose, you have a 2/3 probability of selecting a door that has no prize, and you have a 1/3 chance of selecting the door that has the prize. Of course, the game isn't over then, and we can just think about what happens depending on your choice of strategy. You could, at that juncture, decide to switch. You could decide to not switch. And the same thing happens here. You could decide to switch or not switch.
And the way to think about it is that once I select a door that doesn't have the prize, then the host has no choice but to reveal to me the only other door that doesn't have the prize. And then when I switch, I definitely win. So I win as soon as I select a door in the beginning with no prize if I'm committed to the strategy switch. And if I don't switch, then of course, I lose. And the same thing happens here, but in reverse.
If I got unlucky and I selected the door with the prize and I committed to the strategy switch, then the host would show me one of these two doors without the prize, and I would switch, and I would lose my prize. But this only happens with probability 1/3, because that's the probability that this tree associates with it, and otherwise I'd win. So you can see that once I've committed to the fact that I'm going to switch, all I need to do is I need to get not super lucky, just I need to select the door which doesn't have the prize, and then automatically you can see that I will win.
So we'll do more examples of these kinds of things later, especially in recitation, like drawing out these diagrams to organize the state space. Let me tell you one more really cool tool, and then we'll do some real world applications of it. This involves, again, conditional probability. That's called Bayes' rule. And it's a very basic formula.
So imagine that we compute something the probability of B given A. It turns out there's a way to express this in terms of the conditional probability the other way around. You just have to correct by the right factors of P of B and P of A. So this is a very powerful rule. The proof is quite straightforward. It's just algebraic. Because all we have to do is just invoke the definitions. So by definition, how did we define the probability of A given B?
It was just the probability of A and B divided by the probability of B. That's literally what our definition was. I can take this definition, and I can just multiply by this correction term, which is probability of B over probability of A. And if I do this on both sides of my equation, well, one side I'm going to get this right hand side. So I'll get probability of A given B times probability of B over the probability of A.
And on my other side, the probability of B's are going to cancel. I'm going to get probability of A and B over probability of A. And I claim that this is the same thing as the probability of B given A. And that's because it doesn't matter what order I intersect these two things. Probability of A and B is the same thing
as the probability of B and A. And then this is, again, literally the definition of conditional probability. So the proof of this Bayes rule is very simple. In fact, anyone know anything historically about why people were interested in this? It's kind of an interesting historical fact. Yeah?
Student 33:32 ↗
They wanted to prove God.

7 more exchanges in this transcript

Sign in free to read the rest of this interview. No card required.

Sign in to read the full transcript

Cite this transcript

APA, MLA, BibTeX
APA

Moitra, A. (2025, December 17). Lecture 2: Independence and Conditioning [Interview transcript]. MIT OpenCourseWare. CEOInterviews.AI. https://ceointerviews.ai/interview/3287811/

MLA

Ankur Moitra. "Lecture 2: Independence and Conditioning." MIT OpenCourseWare, 17 Dec. 2025. Transcript, CEOInterviews.AI, https://ceointerviews.ai/interview/3287811/.

BibTeX
@misc{moitra2025_3287811,
  author       = {Ankur Moitra},
  title        = {Lecture 2: Independence and Conditioning},
  howpublished = {Interview transcript, MIT OpenCourseWare. CEOInterviews.AI},
  year         = {2025},
  month        = {dec},
  url          = {https://ceointerviews.ai/interview/3287811/},
  note         = {Speaker-attributed transcript with timestamps}
}