Skip to content

Instantly share code, notes, and snippets.

@Nikolaj-K
Last active November 28, 2024 18:35
Show Gist options
  • Save Nikolaj-K/74add3b5e5118a7c8c260a08c1bfd503 to your computer and use it in GitHub Desktop.
Save Nikolaj-K/74add3b5e5118a7c8c260a08c1bfd503 to your computer and use it in GitHub Desktop.
Bayes' Rule vs Propositional Logic

Script discussed in the video:

https://youtu.be/tqmtZ82WHJc

==== Bayesian calculus vs Propositional logic ==== Idea: Logical implications are akin to Conditional probabilities. Reading: $P(B \vert A)=1 \iff A\to B$ $P(B \vert A)=0 \iff \neg(A\to B)$ "$P(B \vert A)$ ... $P(A\to B)$" Surely $P(\neg A \lor B) \ge P(B)$ Suggests $P(A\to B) \ge P(B)$ Bayes' rule gives proportionality factor $\propto P(B)$ above: $P(B\vert A)\cdot P(A) = P(A\vert B)\cdot P(B)$ So, concretely $P(B\vert A) = \dfrac{P(A\vert B)}{P(A)}\cdot P(B)$ Or $P(B\vert A) = \dfrac{P(A\vert B)}{\sum_H^\text{partition} P(A\vert H)\cdot P(H)}\cdot P(B) = \dfrac{P(A\vert B)}{{\mathbb E}_B[P(A\vert B)]}\cdot P(B)$

And also clearly $P(A\to B) = r_{A,B} \cdot P(B\to A)$


Conditional probabilities as e.g. for probability measures $P\colon {\mathcal P}T\to[0,1], P(T)=1$

$A,B\subset T$ $P(A)\neq 0\ ...\ P(B \vert A) := \dfrac{P(B\cap A)}{P(A)}$ (Defined only when $A$ isn't a null-set, like $A={}$.)

This also Immediately implies Bayes' theorem. Note: If $P(B)\neq 0$, then $P(B \vert A) = \dfrac{P(B\cap A)}{P(A)\cdot P(B)}\cdot P(B)$


Warmup round without the conditional probability notion:

Assuming explosion, we get a propositional version of the Disjunctive Syllogism, $(\neg A\lor B)\to (A\to B)$ With this $\neg (A\to B)\leftrightarrow(\neg \neg A\land \neg B)$ In a bivalent truth table (for which, side note, $\neg \neg A = A$), the right hand side, a conjunction, exactly corresponds to the one failure case of implication.

With the Law of Total probability in some probability theory $P(Q) + P(\neg Q) = 1$ (Compare with LEM, $Q\lor\neg Q$) we got $P(A\to B) = 1 - P(\neg \neg A\land \neg B)$ "There's a chance that $A$ implies $B$, or otherwise we have the one failure case."


Extremal cases:

$\bullet$ $P(T \vert A) = \dfrac{P(T\cap A)}{P(A)} = 1 = P(T)$ ("Trivial conclusion is unconditionally true.") Compare with $(A\to \top) \leftrightarrow \top$

(But note that we ruled out null $A$, i.e. we assumed $P(A)>0$, while the propositional formulas, assuming prove the logical equivalent, here and below, also for false $A$.)

$\bullet$ $P(B \vert T) = \dfrac{P(B\cap T)}{P(T)} = P(B)$ ("Trivial assumption doesn't improve credence.") Also compare with $(\top \to B) \leftrightarrow B$

Inclusion-exclusion: $P(X \cup Y) = P(X) + P(Y) - P(X \cap Y)$ Or $P(X \cap Y) = P(X) - \delta$ with $\delta = P(X \cup Y)-P(Y)$. By monotonicity, $\delta\ge 0$, so $P(Y)=1$ kills off the second terms. $\bullet$ $P(A) = 1\ \to\ P(B \vert A) = P(B)$, generalizing the above. But also $\bullet$ $P(A) = 1\ \to\ P(A \vert B) = 1$ ("Credence of something established to be true is not lowered in light of further assumptions.") Compare $A\to (B\to A)$ (Implication introduction)

Particular to the set approach: $P(B) + P(B^c) = 1$ and further via intersection distributing over unions $P(B \vert A) + P(B^c \vert A) = 1$ (Law of total probability, independent of conditionals.) Compare with $B\lor\neg B$, as noted previously, or also equivalently, the (only formally) stronger $(A\to B)\lor(A\to \neg B)$


So recall $P(B), P(A \vert B) = P(A), P(B\vert A)$

(Classical, arithmetic reasoning) $a,b\in [0,1]$ $\vdash a\cdot b=1\implies a=1\land b=1$ $\vdash a\cdot b=0\implies a=0\lor b=0$

So $\big(B\land(B\to A)\big) \leftrightarrow \big(A\land(A\to B)\big)$ and classically $\big(\neg B\lor\neg (B\to A)\big) \leftrightarrow \big(\neg A\lor\neg (A\to B)\big)$

$P(B)=P(A)>0$, then $P(A \vert B) = P(B\vert A)$. Akin to $(B\leftrightarrow A)\leftrightarrow\big((B\to A)\leftrightarrow(A\to B)\big)$ (Although this is a general tautology)

Studying negations... $P(A \vert \neg B)\cdot P(\neg B) = P(\neg B\vert A)\cdot P(A)$. $P(A \vert \neg B)\cdot \big(1-P(B)\big) = \big(1-P(B\vert A)\big)\cdot P(A)$.

$P(\neg B)>0$: $P(A\vert \neg B) = P(A)\cdot\frac{1-P(B\vert A)}{1 - P(B)}$ $P(\neg A\vert \neg B) = 1 - P(A)\cdot\frac{1-P(B\vert A)}{1 - P(B)}$

$P(A)>0$: $P(B\vert A) = 1\ \leftrightarrow\ P(\neg A\vert \neg B) = 1$ So (under the constraint that we didn't consider the explosive antecedents) $(A \to B) \leftrightarrow (\neg B \to \neg A)$.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment