Machine Learning Mathematics Review-1. Basics of Probability Theory

Probability correlation

Various probability concepts derived from the dice experiment

1. Roll the dice, the probability that the number of points is 6 is $$\frac{1}{6}$$. Roll the dice, known to be an even number , and the probability that the number of points is 6 is $$\frac{1 }{3}$$, this probability is conditional probability .

2. conditional probability is: Suppose we know the A event has occurred, on this basis, we want to know the probability of B events, the probability of conditional probability, denoted by $$ P (B | A) $$

3. classical probability model : Suppose an experiment, there are $$\Omega$$ equal possible results , event A contains $$X$$ of which results, event B contains $$Y$$ of them As a result, $$Z$$ represents the crossover event:

The probability of occurrence of event A: $$P(A) = \frac{X}{\Omega}$$; the probability of occurrence of event B: $$P(B) = \frac{Y}{\Omega}$$; The probability that both events A and B will occur: $$P(AB) = \frac{Z}{\Omega}$$ If event A has occurred, then the probability that event B will also occur is $$P(B|A) = \frac{Z}{X}$$, expand the formula: This formula is the conditional probability formula

$$ P(B|A) = \frac{\frac{Z}{\Omega}}{\frac{X}{\Omega}}= \frac{P(AB)}{P(A)} $$

4. If the conditional probability $$P(B|A)$$ is greater than $$P(B)$$, it means that the occurrence of event A will promote the occurrence of event B, such as the example of throwing the dice above. You can also see the figure below. The probability of $P(B)$$ itself is relatively small. In the case where event A has occurred, due to the large number of intersections, the probability of event B has also increased:

5. If the conditional probability $$P(B|A)$$ is less than $$P(B)$$, it means that event A will not promote the occurrence of event B. For example, event A is an even number of dice rolls, and event B is a roll If the number of dice points is less than <4, the probability of occurrence of event A and event B are both $$1/2$$, the probability of both events A and B occurring at the same time is $$1/6$$, and the conditional probability $$P(B|A)$ $ Is $$1/3$$. You can also see the figure below. The probability of $P(B)$$ itself is relatively large. In the case where event A has occurred, the probability of event B occurring is reduced due to the small number of intersections:

6. If the conditional probability $$P(B|A)$$ is equal to 0, it means that event A and event B are completely disjoint, that is, if event A occurs, event B must not occur, and event A and event B are incompatible Event , or mutually exclusive event . As shown below:

7. It is also possible that the conditional probability $$P(B|A)$$ is equal to $$P(B)$$. In this case, the occurrence of events A and B are not related to each other. For example, there are two dice, Event A rolls the number 6 for dice 1, and event B rolls the number 2 for dice 2. The probability of event A and event B are both $$1/6$$, then the probability of events A and B occurring at the same time is $$\ frac{1}{36}$$, the conditional probability $$P(B|A)$$ is equal to $$\frac{1}{6}$$, we generally call this kind of independent event. As shown below:

Total probability formula and dice experiment verification

Suppose there are these mutually exclusive events $$A_1,A_2,...,A_n$$, which contain all possible results of the experiment:

That is, $$P(A_1) + P(A_2) + ... + P(A_n) = 1$$. Take the dice just now as an example. In fact, it is to throw the dice once, and the points are 1, 2, 3, 4, 5, 6.

Suppose there is another event B, represented by classical probability as shown in the figure:

The probability of event B can be calculated by the conditional probability of event B on these mutually exclusive events $$A_1,A_2,...,A_n$$ and the probability of these events, that is, the total probability formula:

$$ 条件：P(A_1) + P(A_2) + ... + P(A_n) = 1 $$

$$ 结果：P(B) = P(B\Omega) = P(BA_1) + P(BA_2) + ... + P(BA_n) = P(A_1)P(B|A_1) + P(A_2)P(B|A_2) + ... + P(A_n)P(B|A_n) $$

For example, in event B, the dice cast is an even number, $$P(B) = \frac{1}{2}$$, $$P(A_{points=1})P(B|A_{points=1} ) + P(A_{Points=2})P(B|A_{Points=2}) + P(A_{Points=3})P(B|A_{Points=3}) + P(A_{Points= 4})P(B|A_{Points=4}) + P(A_{Points=5})P(B|A_{Points=5}) + P(A_{Points=6})P(B|A_ {Points=6}) = \frac{1}{6} * 0 + \frac{1}{6} * 1 + \frac{1}{6} * 0 + \frac{1}{6} * 1 + \frac{1}{6} * 0 + \frac{1}{6} * 1 = \frac{1}{2}$$

Use of the full probability formula: football prediction

Meaning that the total probability formula: in most cases, we are probability as difficult as the dice experiment directly derived event B , we need to define the sample space event, $$ A_1 based on existing sample abstract event ,A_2,...,A_n$$, meanwhile count the probability of occurrence of B on these events, and finally get the probability of event B.

For example, to speculate on the probability of the British team winning against the German team in this European Cup. We can estimate the British team's goals based on historical game data (such as data from recent European Cup games and the data of the two teams against each other). The probability that the number is 0,1,2,3,4,5..., the probability that the German team scores 0,1,2,3,4,5..., the number of goals British team is greater than The probability of victory for the German team, the British team, is . This is an application of the total probability formula.

Infer the effect from the cause and infer the cause from the effect

Total probability formula is by the push because if speculated that a typical example is the above-mentioned European Cup England against Germany, England probability of victory. Based on previous game data, we can calculate the average goals scored by the British team and the German team. The probability of scoring generally conforms to the Poisson distribution (this will be mentioned later, and we will use this example for detailed analysis). According to Loose distribution , we can get the probability that the British team and the German team score n, assuming that the British team’s average goal is 1.67 and the German team’s average goal is 1.52 (we only consider the number of goals here is 4. Case):

team	0 goals	The number of goals is 1	The number of goals is 2	3 goals	The number of goals is 4
British team	0.1882	0.3144	0.2625	0.1461	0.061
German team	0.2187	0.3324	0.2527	0.128	0.0486

Suppose $$P(A_0)$$ is the probability that the number of goals scored by the British team is 0 and so on:

$$ P(A_0) = 0.1882 $$

$$ P(A_1) = 0.3144 $$

$$ P(A_2) = 0.2625 $$

$$ P(A_3) = 0.1461 $$

$$ P(A_4) = 0.061 $$

Assuming $$P(B)$$ is the probability of the British team winning, according to the total probability formula:

$$ P(B) = P(A_0)P(B|A_0) + P(A_1)P(B|A_1) + P(A_2)P(B|A_2) + P(A_3)P(B|A_3) + P(A_4)P(B|A_4) $$

$$ P(B|A_0) = 0 $$

$$ P(B|A_1) = 德国队进球为 0 的概率 = 0.2187 $$

$$ P(B|A_2) = 德国队进球为 0，1 的概率 = 0.2187 + 0.3324 = 0.5511 $$

$$ P(B|A_3) = 德国队进球为 0，1，2 的概率 = 0.2187 + 0.3324 + 0.2527 = 0.8038 $$

$$ P(B|A_4) = 德国队进球为 0，1，2，3 的概率 = 0.2187 + 0.3324 + 0.2527 + 0.128 = 0.9318 $$

$$ P(B) = 0.1882 * 0 + 0.3144 * 0.2187 + 0.2625 * 0.5511 + 0.1461 * 0.8038 + 0.061 * 0.9318 = 0.3877 $$

However, in practical problems, we often encounter problems caused by effect. For example, when our physical examination detects gallbladder polyps, is it caused by tumors, cholesterol, or other reasons? This requires us to speculate on the reasons for the formation from this result. This leads to the Bayesian formula

Understanding prior probability and posterior probability from football prediction examples

Before mentioning the Bayesian formula, let's clarify two concepts, prior probability and posterior probability .

prior probability of generally obtained through experience, that is, the empirical probability obtained based on historically collected data without any restrictions. In the above example, the probability of the number of goals between the two teams inferred from historical game data is priori probability of . At this time, suppose that the game started, and then an incident occurred. The German defender made a mistake and was advanced by the British team Kane. At this time, we need to recalculate the probability of the number of goals scored by the two teams under this premise. This is the posterior probability of .

prior probability of is the empirical probability that is completely inferred from historical data, and there is no probability under the condition that has occurred. posterior probability that is required to observe a phenomenon prior probability correct probability. It can be simply understood in this way. Before the start of the game, the estimated probability is generally the prior probability. After the game starts, after red and yellow cards, penalty kicks, goals, substitutions, etc. occur, the probability is corrected after the probability is obtained. Probability of experience.

Bayesian formula and speculation on the cause of gallbladder polyps

Assuming there are events A and B, then:

$$ P(A|B) = \frac{P(AB)}{P(B)} = \frac{P(B|A)P(A)}{P(B)} $$

This is the Bayesian formula. Let's combine the total probability formula. Assuming that our events $$A_1, A_2, ..., A_n$$ these mutually exclusive events constitute the complete set of sample space, we have:

$$ P(A_1|B) = \frac{P(B|A_1)P(A_1)}{P(B)} = \frac{P(B|A_1)P(A_1)}{P(B|A_1)P(A_1) + P(B|A_2)P(A_2) + ... + P(B|A_n)P(A_n)} $$

Let’s use the cause of gallbladder polyps to speculate. For example, suppose we count that in a sample of one million patients in a hospital, 8% of them have tumors, and 20% of them have been found to have gallbladder polyps with symptoms of high cholesterol. There are 80% of people, 40% of them have found gallbladder polyps, and 30% of the remaining 12% have found gallbladder polyps. Suppose $$A_1$$ is suffering from tumor, $$A_2$$ is cholesterol, and $$A_3$$ is others. $$B$$ is the polyp of the gallbladder. Then the probability that the gallbladder polyp is a tumor is:

$$ P(A_1|B) = \frac{P(B|A_1)P(A_1)}{P(B|A_1)P(A_1) + P(B|A_2)P(A_2) + P(B|A_3)P(A_3)} = \frac{0.2 * 0.08}{0.2 * 0.08 + 0.4 * 0.8 + 0.3 * 0.12} = 0.043 $$