{ "format": "recallweave-deck/1", "title": "Markov chains: where probability goes next", "attribution": "Original questions, numerical examples and worked transfers by RecallWeave contributors with AI assistance, 2026. Mathematical background checked against MIT OpenCourseWare: 6.436J/15.085J, Fall 2018, Lecture 21, Markov Chains (https://ocw.mit.edu/courses/6-436j-fundamentals-of-probability-fall-2018/141423989a49375f14f0a44940d81e67_MIT6_436JF18_lec21.pdf); Hao Wu, 18.445, Spring 2015, Lectures 2 and 4 (https://ocw.mit.edu/courses/18-445-introduction-to-stochastic-processes-spring-2015/pages/lecture-notes/). No source exercises, prose or figures are reproduced. All systems here are hypothetical teaching examples.", "license": "CC0-1.0 for this original course content. Referenced MIT materials retain their own terms and are not included in this deck.", "concepts": [ "State and transition rules", "Probability flow", "Repeated steps and observations", "Stationary distributions", "Cycles and closed states" ], "items": [ { "id": "mc-rule", "concept": "State and transition rules", "prerequisites": [], "prompt": "A time-homogeneous Markov model uses states A, B and C. After the current state is known, which information determines its modeled probabilities for the next state?", "options": [ "Only the current state, through the same transition rule at every step.", "The exact earlier path, even when the current state is the same.", "Only the number of steps already taken; the current state is irrelevant.", "A requirement that the next state be different from the current state." ], "answer": 0, "explanation": "The Markov assumption says that, conditional on the present state, the earlier path adds no information about the next-state probabilities. Time-homogeneous means that this conditional rule does not change with the step number. It does not make successive states independent, and self-transitions are allowed.", "transfer": "A machine's failure chance depends on both its current mode and its age. Would a model using only the mode automatically be Markov? No. Age may contain relevant information not captured by that state. Adding a suitable age category could improve the state description; whether it is adequate still needs evidence." }, { "id": "mc-row", "concept": "State and transition rules", "prerequisites": [], "prompt": "Rows name the current state; columns name the next state in order A, B, C. The row for A is [60%, 20%, 20%]. What does its middle entry mean?", "options": [ "Exactly 20 of every 100 transitions in a finite run will end in B.", "The system spends 20% of all future time in A.", "Given that the current state is A, the modeled chance that the next state is B is 20%.", "Given that the next state is B, the chance the current state was A is 20%." ], "answer": 2, "explanation": "A row gives conditional next-state probabilities for one specified current state. The middle entry is P(next = B | current = A). Reversing the conditioning generally needs the current distribution and other transition probabilities. A probability also does not force exact counts in a finite observed run.", "transfer": "If the B row is [40%, 50%, 10%], what is the chance of A next when the current state is B? It is 40%. The A-to-B chance remains 20%; opposite directions need not have equal probabilities." }, { "id": "mc-valid-row", "concept": "State and transition rules", "prerequisites": [], "prompt": "A row lists all possible next states A, B and C. Which row is a valid probability row without changing any entry?", "options": [ "[60%, 30%, 20%]", "[-10%, 60%, 50%]", "[0%, 0%, 0%]", "[0%, 35%, 65%]" ], "answer": 3, "explanation": "Every entry must be nonnegative and the complete row must sum to 100%. The last row meets both conditions. The other rows respectively sum to 110%, contain a negative probability, and omit all probability mass. A zero entry is allowed; a row of zeros is not.", "transfer": "Does a row [100%, 0%, 0%] need to be smoothed to be valid? No. It describes a certain move to A. Whether that is a useful model is a separate question. In the explorer, invalid rows are reported instead of silently normalized." }, { "id": "mc-mixture", "concept": "Probability flow", "prerequisites": [ "State and transition rules" ], "prompt": "Use the transition rows A:[60%,20%,20%], B:[40%,50%,10%], C:[40%,10%,50%], with columns A,B,C. The current distribution is [50%,50%,0%]. What is the next distribution?", "options": [ "[60%,20%,20%]", "[50%,35%,15%]", "[100%,70%,30%]", "[50%,50%,0%]" ], "answer": 1, "explanation": "Weight each row by the probability of its current state. A next receives 0.50×0.60 + 0.50×0.40 = 0.50. B receives 0.50×0.20 + 0.50×0.50 = 0.35. C receives 0.50×0.20 + 0.50×0.10 = 0.15. Contributions from current C are zero. The result sums to one.", "transfer": "Change the current distribution to [0%,50%,50%]. The next distribution is [40%,30%,30%]: A gets 0.50×0.40 twice; B gets 0.50×0.50 + 0.50×0.10; C gets 0.50×0.10 + 0.50×0.50." }, { "id": "mc-two-steps", "concept": "Repeated steps and observations", "prerequisites": [ "Probability flow" ], "prompt": "Use the same fixed rows A:[60%,20%,20%], B:[40%,50%,10%], C:[40%,10%,50%]. Start certainly in A at step 0. Step 1 is [60%,20%,20%]. What is the distribution at step 2?", "options": [ "[52%,24%,24%]", "[36%,4%,4%]", "[60%,20%,20%]", "[50%,25%,25%]" ], "answer": 0, "explanation": "Apply the full transition rule again. A receives 0.60×0.60 + 0.20×0.40 + 0.20×0.40 = 0.52. B receives 0.60×0.20 + 0.20×0.50 + 0.20×0.10 = 0.24. C receives 0.60×0.20 + 0.20×0.10 + 0.20×0.50 = 0.24. Squaring entries individually drops paths through intermediate states.", "transfer": "One more step gives [50.4%,24.8%,24.8%]. For A, calculate 0.52×0.60 + 0.24×0.40 + 0.24×0.40 = 0.504. Repeated multiplication describes repeated use of the same conditional rule." }, { "id": "mc-path-versus-endpoint", "concept": "Probability flow", "prerequisites": [ "State and transition rules" ], "prompt": "The fixed rows are A:[60%,20%,20%], B:[40%,50%,10%], C:[40%,10%,50%]. Starting in A, what is the probability of the particular two-step path A → B → C?", "options": [ "24%, because that is the total probability of ending in C after two steps.", "30%, obtained by adding the two transition probabilities.", "2%, obtained from 20% × 10%.", "10%, because only the last transition matters." ], "answer": 2, "explanation": "The specified path requires both transitions: A to B with probability 0.20, then B to C with probability 0.10. Their conditional product is 0.02. Ending in C after two steps also allows A→A→C and A→C→C; summing all three disjoint paths gives 0.12 + 0.02 + 0.10 = 0.24.", "transfer": "What is the probability of A → C → A? It is 0.20×0.40 = 0.08, or 8%. This is only one contribution to the full step-2 probability of A, which is 52%." }, { "id": "mc-distribution-not-count", "concept": "Repeated steps and observations", "prerequisites": [ "Probability flow" ], "prompt": "A toy model gives the step-2 distribution [52%,24%,24%]. One hundred independent copies use that model and all start in A. Which conclusion is justified?", "options": [ "Every copy must pass through all three states by step 2.", "Exactly 52 copies will be in A in this particular experiment.", "Each copy must spend 52% of its first two steps in A.", "The expected number in A at step 2 is 52, while the observed number can differ." ], "answer": 3, "explanation": "For each copy, the probability of being in A at that specified step is 0.52. Summing the 100 indicator expectations gives an expected count of 52. An expectation need not equal a finite experiment's realized count. A distribution at one step is also different from a time fraction along one path.", "transfer": "For 25 independent copies, the expected number in A is 25×0.52 = 13. An observed count such as 11 does not by itself contradict the model. The explorer computes distributions directly and does not generate a random sample of paths." }, { "id": "mc-step-zero", "concept": "Repeated steps and observations", "prerequisites": [ "Probability flow" ], "prompt": "An initial distribution is [20%,30%,50%]. Before applying any transition, what should the explorer display at step 0?", "options": [ "[0%,0%,0%], because no transitions have occurred.", "[20%,30%,50%], the initial distribution itself.", "The first row of the transition matrix.", "The stationary distribution, whatever the initial distribution was." ], "answer": 1, "explanation": "Step 0 describes the initial state uncertainty. Applying zero transitions leaves it unchanged. A transition matrix changes the distribution only when a step is taken; it does not replace the initial condition automatically.", "transfer": "If the initial distribution is [0%,100%,0%], the system is certainly in B at step 0. Its step-1 distribution is the B row, not the A row and not an average of all rows." }, { "id": "mc-stationary-test", "concept": "Stationary distributions", "prerequisites": [ "Probability flow" ], "prompt": "A candidate vector π has nonnegative entries summing to one. In the row-vector convention p(next) = p(current)P, what checks whether π is stationary?", "options": [ "All entries of π must be equal.", "Every transition in P must leave its state unchanged.", "Multiplying once must give πP = π.", "A chart must look nearly flat after a finite number of plotted steps." ], "answer": 2, "explanation": "Stationary means that starting with distribution π produces exactly the same distribution after a transition. Then repeated transitions preserve it as well. The condition concerns the distribution, not whether individual paths move. Uniform probabilities and a visually flat finite plot are not general definitions of stationarity.", "transfer": "For rows A:[60%,20%,20%], B:[40%,50%,10%], C:[40%,10%,50%], try π=[50%,25%,25%]. The next A probability is 0.30+0.10+0.10=0.50; B is 0.10+0.125+0.025=0.25; C is 0.10+0.025+0.125=0.25." }, { "id": "mc-stationary-does-not-freeze", "concept": "Stationary distributions", "prerequisites": [ "Probability flow" ], "prompt": "Use rows A:[0%,50%,50%], B:[100%,0%,0%], C:[100%,0%,0%]. The initial distribution [50%,25%,25%] is stationary. What happens to an individual state on its next transition?", "options": [ "It stays in its current state, because the distribution is stationary.", "It changes state with probability one, even though the overall distribution stays the same.", "It must become A with probability one, regardless of its current state.", "Its next state is independent of its current state." ], "answer": 1, "explanation": "Every diagonal entry is zero. From A the next state is B or C, and from B or C it is A, so every path changes state. Meanwhile the mass in B and C totals 50% and flows into A; A's 50% splits into 25% for B and 25% for C. The distribution is unchanged without freezing individuals.", "transfer": "Starting from that stationary distribution, the probability of being in A is 50% at every step. Starting certainly in A instead, it is 100% at step 0 and 0% at step 1. Stationarity is a property of a particular distribution together with the transition rule." }, { "id": "mc-alternation", "concept": "Cycles and closed states", "prerequisites": [ "Repeated steps and observations", "Stationary distributions" ], "prompt": "Use A:[0%,50%,50%], B:[100%,0%,0%], C:[100%,0%,0%], and start certainly in A. Which description is correct?", "options": [ "The probability of A is 100% at every even step and 0% at every odd step, so these distributions do not converge.", "The distribution becomes [50%,25%,25%] after two steps and stays there.", "There is no stationary distribution because the path keeps moving.", "Increasing the plotted horizon must eventually remove the alternation." ], "answer": 0, "explanation": "One step sends all mass from A into B and C equally. The next sends all of it back to A. This repeats forever: [1,0,0], then [0,0.5,0.5], then [1,0,0]. The chain has a stationary distribution [0.5,0.25,0.25], but the distributions from this initial state do not approach it. Stationarity alone does not guarantee convergence.", "transfer": "At step 9 the distribution is [0%,50%,50%]; at step 10 it is [100%,0%,0%]. Averaging many steps is a different operation from asking for the distribution at one late step. A longer plot does not change the rule." }, { "id": "mc-mixing-conditions", "concept": "Cycles and closed states", "prerequisites": [ "Repeated steps and observations", "Stationary distributions" ], "prompt": "Consider a finite, time-homogeneous Markov chain. Every state can reach every other (irreducible), and the chain has period one (aperiodic). What follows?", "options": [ "All states have equal stationary probabilities.", "Every transition becomes deterministic after enough steps.", "Every finite observed path visits each state equally often.", "There is a unique stationary distribution, and distributions from any initial distribution approach it as the number of steps grows." ], "answer": 3, "explanation": "For a finite irreducible aperiodic chain, the convergence theorem gives a unique stationary distribution and convergence to it from any initial distribution. It does not require a uniform stationary distribution or exact agreement after a finite number of steps. Irreducibility alone guarantees uniqueness for a finite chain, but a periodic chain can still fail to converge from some starts.", "transfer": "The mixing preset has rows [60%,20%,20%], [40%,50%,10%], [40%,10%,50%]. Every transition is positive, so all states communicate and each has a self-return in one step. Its stationary distribution is [50%,25%,25%], which is not uniform. A rounded chart near that value is an illustration, not a proof." }, { "id": "mc-absorbing", "concept": "Cycles and closed states", "prerequisites": [ "State and transition rules", "Repeated steps and observations" ], "prompt": "Rows and columns use order A, B, C. In a toy model, the A row is [100%,0%,0%]. What can you conclude from that row alone?", "options": [ "Every initial state reaches A after at most one step.", "A is absorbing: once the state is A, all later states remain A under this unchanged rule.", "A is the only possible stationary state for every choice of the B and C rows.", "The current distribution must assign 100% probability to A." ], "answer": 1, "explanation": "The row guarantees a self-transition from A with probability one, making A absorbing. It says nothing by itself about whether B or C can reach A. They could have their own closed behavior. The current distribution is an initial or evolved probability vector, not something imposed by the A row alone.", "transfer": "If B's row is [0%,100%,0%] too, starting in B never reaches A. In the explorer's absorbing preset, B and C do have routes into A; the gradually increasing A probability illustrates those additional rows, not just the fact that A is absorbing." }, { "id": "mc-closed-start", "concept": "Cycles and closed states", "prerequisites": [ "Repeated steps and observations", "Stationary distributions" ], "prompt": "Use A:[100%,0%,0%], B:[0%,100%,0%], C:[40%,60%,0%]. What happens when the system starts certainly in C?", "options": [ "It converges to the same distribution as a system starting certainly in A.", "It alternates between A and B with each step.", "After one step its distribution is [40%,60%,0%], which stays unchanged; starting certainly in A instead stays [100%,0%,0%].", "It has no stationary distribution because C is not absorbing." ], "answer": 2, "explanation": "C sends 40% of its mass into absorbing A and 60% into absorbing B. Thereafter each part stays where it went. Starting in A has a different outcome because there is no route from A to B. The chain has more than one stationary distribution; every distribution supported only on A and B is stationary.", "transfer": "Start with [20%,30%,50%]. One step gives A: 0.20 + 0.50×0.40 = 0.40, B: 0.30 + 0.50×0.60 = 0.60, C: 0. These probabilities then stay fixed. This example shows why the initial distribution can keep mattering." } ] }