Skip to content
Back to Modules
Part IA Michaelmas, Lent Term

Discrete Mathematics

Mathematics Logic Proofs Sets Relations Functions Formal-Languages Automata Cardinality Part IA Michaelmas Term Lent Term
  • Mathematical Proofs

    Foundations of logic, direct proofs, induction, GCD, Fermat's Little Theorem, and modular arithmetic

    • Statements and Mathematical Jargon

      What is a Statement?

      A statement is a sentence that is either true or false, but not both. This is the fundamental unit of mathematical discourse.

      Examples of Statements

      • eiπ+1=0e^{i\pi} + 1 = 0
      • “17 is a prime number”
      • “There are infinitely many primes”

      Non-examples

      • “This statement is false” (paradox)
      • “Is 7 prime?” (question, not a statement)
      • “Let x=5x = 5” (command, not a statement)

      Predicates

      A predicate is a statement whose truth depends on the value of one or more free variables.

      Examples

      • P(n):P(n) : \equivnn is even”
      • Q(x,y):Q(x, y) : \equiv "x2+y2=1x^2 + y^2 = 1"
      • “The function ff is differentiable”

      A predicate becomes a statement once we substitute specific values for its free variables.

      The Hierarchy of Mathematical Results

      Mathematicians distinguish different types of true statements:

      Axiom

      A statement assumed true without proof. Axioms form the foundation of a mathematical theory.

      Example: The axioms of Zermelo-Fraenkel set theory (ZF).

      Definition

      A name given to a concept, establishing terminology.

      Example: “A prime number is a natural number greater than 1 divisible only by 1 and itself.”

      Theorem

      A major proved result of significant importance.

      Example: Fermat’s Last Theorem, the Fundamental Theorem of Arithmetic.

      Proposition

      A proved result of lesser importance than a theorem.

      Lemma

      A result proved chiefly to support the proof of another result. Lemmas are “helper theorems.”

      Example: The “lifting the exponent lemma” used in number theory proofs.

      Corollary

      A statement that follows easily from a previously proved theorem or proposition.

      Example: “There are infinitely many primes congruent to 3 mod 4” is a corollary of Dirichlet’s theorem.

      Conjecture

      A statement believed to be true but not yet proved.

      Example: The Goldbach Conjecture (every even integer greater than 2 is the sum of two primes).

      Why Precision Matters

      Consider the statement: “The product of two odd integers is odd.”

      This hides significant logical structure. More precisely:

      For all integers m,n: if m and n are odd, then mn is odd.\text{For all integers } m, n: \text{ if } m \text{ and } n \text{ are odd, then } m \cdot n \text{ is odd.}

      Even more formally:

      m,nZ.(Odd(m)Odd(n))Odd(mn)\forall m, n \in \mathbb{Z}. \left( \text{Odd}(m) \land \text{Odd}(n) \right) \Rightarrow \text{Odd}(m \cdot n)

      Tripos Credit

      Marks are awarded for making such logical structure explicit, not just for getting the right answer. A solution that correctly proves the result but fails to exhibit awareness of the quantifiers and implication structure may lose marks.

      Variables and Scope

      Free Variables

      A variable is free in a statement if the statement’s truth depends on its value.

      In "x>5x > 5", the variable xx is free.

      Bound Variables

      A variable is bound if it is quantified by \forall or \exists.

      In "xR.x20\forall x \in \mathbb{R}. x^2 \geq 0", the variable xx is bound by the universal quantifier.

      Common Notation

      Logical Symbols

      SymbolMeaning
      \landand (conjunction)
      \loror (disjunction)
      ¬\negnot (negation)
      \Rightarrowimplies
      \Leftrightarrowif and only if
      \forallfor all
      \existsthere exists

      Set Notation

      SymbolMeaning
      \inelement of
      \subseteqsubset
      \cupunion
      \capintersection
      \setminusset difference
      N\mathbb{N}natural numbers {0,1,2,}\{0, 1, 2, \ldots\}
      Z\mathbb{Z}integers {,1,0,1,}\{\ldots, -1, 0, 1, \ldots\}
      Q\mathbb{Q}rational numbers
      R\mathbb{R}real numbers

      Number Notation

      • aba \mid b means ”aa divides bb
      • ab(modn)a \equiv b \pmod{n} means ”aa is congruent to bb modulo nn
      • gcd(a,b)\gcd(a, b) means “greatest common divisor of aa and bb

      Summary

      • A statement is a sentence that is either true or false
      • A predicate is a statement with free variables
      • Mathematical results form a hierarchy: axiom, definition, theorem, proposition, lemma, corollary, conjecture
      • Precision in stating logical structure is essential for rigorous proof and earns Tripos credit
    • Implication and Direct Proof

      The Implication Connective

      An implication is a statement of the form “If PP, then QQ”, written PQP \Rightarrow Q.

      • PP is the antecedent (or hypothesis, premise)
      • QQ is the consequent (or conclusion)

      Truth Table

      PPQQPQP \Rightarrow Q
      TTT
      TFF
      FTT
      FFT

      The implication PQP \Rightarrow Q is false only when PP is true and QQ is false.

      Vacuous Truth

      When PP is false, the implication PQP \Rightarrow Q is true regardless of QQ. This is called vacuous truth.

      Example: “If 0=10 = 1, then pigs fly” is true (vacuously).

      Logical Equivalences

      Contrapositive

      The contrapositive of PQP \Rightarrow Q is ¬Q¬P\neg Q \Rightarrow \neg P.

      Theorem: An implication is logically equivalent to its contrapositive: PQ¬Q¬PP \Rightarrow Q \equiv \neg Q \Rightarrow \neg P

      Converse

      The converse of PQP \Rightarrow Q is QPQ \Rightarrow P.

      Important: The converse is NOT equivalent to the original. Proving the converse does not prove the original implication.

      PPQQPQP \Rightarrow QQPQ \Rightarrow P
      TTTT
      TFFT
      FTTF
      FFTT

      Inverse

      The inverse of PQP \Rightarrow Q is ¬P¬Q\neg P \Rightarrow \neg Q.

      Important: The inverse is equivalent to the converse (not to the original).

      Proof Strategies for Implication

      Method 1: Direct Proof

      To prove PQP \Rightarrow Q:

      1. Assume PP is true
      2. Using logical steps, derive QQ

      Method 2: Proof by Contrapositive

      To prove PQP \Rightarrow Q:

      1. Assume ¬Q\neg Q is true
      2. Using logical steps, derive ¬P\neg P

      The contrapositive is useful when ¬Q\neg Q provides a “stronger” starting point than PP.

      Example: Direct Proof

      Statement: If nn is even, then n2n^2 is even.

      Direct proof:

      1. Assume nn is even
      2. Then n=2kn = 2k for some integer kk
      3. So n2=(2k)2=4k2=2(2k2)n^2 = (2k)^2 = 4k^2 = 2(2k^2)
      4. Since 2k22k^2 is an integer, n2n^2 is even

      Example: Proof by Contrapositive

      Statement: If n2n^2 is even, then nn is even.

      Contrapositive: If nn is odd, then n2n^2 is odd.

      Proof:

      1. Assume nn is odd
      2. Then n=2k+1n = 2k + 1 for some integer kk
      3. So n2=(2k+1)2=4k2+4k+1=2(2k2+2k)+1n^2 = (2k+1)^2 = 4k^2 + 4k + 1 = 2(2k^2 + 2k) + 1
      4. Since 2k2+2k2k^2 + 2k is an integer, n2=2m+1n^2 = 2m + 1 for some integer mm
      5. Therefore n2n^2 is odd

      By the contrapositive, if n2n^2 is even, then nn is even.

      Combining Implications

      Transitivity

      If PQP \Rightarrow Q and QRQ \Rightarrow R, then PRP \Rightarrow R.

      Hypothetical Syllogism

      From PQP \Rightarrow Q and QRQ \Rightarrow R, conclude PRP \Rightarrow R.

      Common Pitfalls

      Confusing Converse and Implication

      Trap: Proving QPQ \Rightarrow P does not prove PQP \Rightarrow Q.

      Example: “If x>5x > 5, then x>3x > 3” is true. Its converse “If x>3x > 3, then x>5x > 5” is false (consider x=4x = 4).

      Assuming the Conclusion

      Trap: When proving PQP \Rightarrow Q, do not assume QQ at the start. This is circular reasoning.

      Forgetting the Antecedent

      Trap: When proving PQP \Rightarrow Q, you must begin by assuming PP. If you forget this, your proof structure is incorrect.

      Multiple Antecedents

      Implications can have multiple antecedents:

      P1P2P3QP_1 \land P_2 \land P_3 \Rightarrow Q

      To prove this:

      1. Assume all of P1P_1, P2P_2, P3P_3
      2. Derive QQ

      Summary

      • PQP \Rightarrow Q means “if PP then QQ
      • The contrapositive ¬Q¬P\neg Q \Rightarrow \neg P is logically equivalent
      • The converse QPQ \Rightarrow P is NOT equivalent
      • Direct proof: assume PP, derive QQ
      • Proof by contrapositive: assume ¬Q\neg Q, derive ¬P\neg P
    • Bi-implication

      The Bi-implication Connective

      A bi-implication is a statement of the form ”PP if and only if QQ”, written PQP \Leftrightarrow Q or ”PP iff QQ”.

      Truth Table

      PPQQPQP \Leftrightarrow Q
      TTT
      TFF
      FTF
      FFT

      PQP \Leftrightarrow Q is true exactly when PP and QQ have the same truth value.

      Logical Equivalence

      PQ(PQ)(QP)P \Leftrightarrow Q \equiv (P \Rightarrow Q) \land (Q \Rightarrow P)

      This decomposition is crucial for proofs.

      Proof Strategy for Bi-implication

      To prove PQP \Leftrightarrow Q, you must prove both directions:

      1. Forward direction: Prove PQP \Rightarrow Q
      2. Reverse direction: Prove QPQ \Rightarrow P

      These two proofs are typically separate and may use different techniques.

      Example

      Statement: An integer nn is even if and only if n2n^2 is even.

      Proof:

      ()(\Rightarrow) If nn is even, then n2n^2 is even:

      • If nn is even, then n=2kn = 2k for some integer kk
      • So n2=4k2=2(2k2)n^2 = 4k^2 = 2(2k^2)
      • Since 2k22k^2 is an integer, n2n^2 is even

      ()(\Leftarrow) If n2n^2 is even, then nn is even:

      • By contrapositive: if nn is odd, then n2n^2 is odd
      • If n=2k+1n = 2k + 1 for some integer kk
      • Then n2=(2k+1)2=4k2+4k+1=2(2k2+2k)+1n^2 = (2k+1)^2 = 4k^2 + 4k + 1 = 2(2k^2+2k) + 1
      • This is odd, so by contrapositive the claim holds

      Therefore nn is even if and only if n2n^2 is even.

      Alternative Proofs

      Sometimes both directions can be proved simultaneously using logical equivalences.

      Example

      Statement: PQP \Rightarrow Q is logically equivalent to ¬PQ\neg P \lor Q.

      Proof:

      Compare truth tables:

      PPQQ¬P\neg PPQP \Rightarrow Q¬PQ\neg P \lor Q
      TTFTT
      TFFFF
      FTTTT
      FFTTT

      Since both columns are identical, PQ¬PQP \Rightarrow Q \Leftrightarrow \neg P \lor Q.

      Chain of Bi-implications

      For a chain of equivalences:

      P1P2P3PnP_1 \Leftrightarrow P_2 \Leftrightarrow P_3 \Leftrightarrow \cdots \Leftrightarrow P_n

      you can prove:

      P1P2P3PnP1P_1 \Rightarrow P_2 \Rightarrow P_3 \Rightarrow \cdots \Rightarrow P_n \Rightarrow P_1

      This proves all pairwise equivalences by transitivity.

      Example

      Statement: x=1x2=1x = 1 \Leftrightarrow x^2 = 1 (for real xx).

      This is false! Check: x=1x = -1 gives x2=1x^2 = 1 but x1x \neq 1.

      Correct statement: For real x0x \geq 0, x=1x2=1x = 1 \Leftrightarrow x^2 = 1.

      Common Pitfalls

      Proving Only One Direction

      Trap: Proving PQP \Rightarrow Q does not prove PQP \Leftrightarrow Q.

      You must prove both directions separately.

      Assuming Equivalence

      Trap: Do not assume PQP \Leftrightarrow Q before proving it.

      If you use logical equivalences in scratch work, verify each step is truly an equivalence.

      Missing Cases

      Trap: A bi-implication over a domain may fail for certain values.

      Always check the domain: ”nn is prime” \Leftrightarrownn has exactly two divisors” fails for n=1n = 1.


      Summary

      • PQP \Leftrightarrow Q means ”PP if and only if QQ
      • Equivalent to (PQ)(QP)(P \Rightarrow Q) \land (Q \Rightarrow P)
      • Prove both directions separately
      • Check that the domain is correct for the equivalence to hold
    • Universal Quantification

      The Universal Quantifier

      The universal quantifier \forall means “for all” or “for every”.

      The statement x.P(x)\forall x. P(x) means: P(x)P(x) is true for every element xx in the domain of discourse.

      Syntax

      xS.P(x)\forall x \in S. P(x)

      means “for all xx in the set SS, P(x)P(x) holds.”

      Examples

      • nN.n0\forall n \in \mathbb{N}. n \geq 0 (true)
      • nZ.n20\forall n \in \mathbb{Z}. n^2 \geq 0 (true)
      • nZ.n>0\forall n \in \mathbb{Z}. n > 0 (false: 1-1 is a counterexample)

      Proof Strategy: Universal Generalisation

      To prove xS.P(x)\forall x \in S. P(x):

      1. Fix an arbitrary xx: “Let xSx \in S be arbitrary”
      2. Prove P(x)P(x): Show that P(x)P(x) holds for this fixed xx
      3. Conclude: Since xx was arbitrary, the statement holds for all xSx \in S

      Key Point

      The xx you work with must be arbitrary, not a specific value. You cannot assume anything about xx beyond what every element of SS satisfies.

      Example

      Statement: For all integers nn, if nn is even then n+2n + 2 is even.

      Proof:

      Let nZn \in \mathbb{Z} be arbitrary. Assume nn is even.

      Then n=2kn = 2k for some integer kk.

      We have n+2=2k+2=2(k+1)n + 2 = 2k + 2 = 2(k + 1).

      Since k+1k + 1 is an integer, n+2n + 2 is even.

      Since nn was arbitrary, the statement holds for all integers.

      Nested Quantifiers

      Multiple quantifiers can be nested. The order matters:

      x.y.P(x,y)\forall x. \forall y. P(x, y)

      “For all xx and for all yy, P(x,y)P(x, y) holds.”

      y.x.P(x,y)\forall y. \forall x. P(x, y)

      These are equivalent: x.y.P(x,y)y.x.P(x,y)\forall x. \forall y. P(x, y) \equiv \forall y. \forall x. P(x, y)

      However, when mixing \forall and \exists, order matters critically.

      Scope and Binding

      Example

      In xN.x+y>0\forall x \in \mathbb{N}. x + y > 0:

      • xx is bound by \forall
      • yy is free (the statement depends on yy)

      In x.y.x+y>0\forall x. \forall y. x + y > 0:

      • Both xx and yy are bound

      Restricted Quantification

      Instead of xS.P(x)\forall x \in S. P(x), you may see:

      x.(xSP(x))\forall x. (x \in S \Rightarrow P(x))

      These are logically equivalent.

      Example

      nN.n0\forall n \in \mathbb{N}. n \geq 0 is equivalent to n.(nNn0)\forall n. (n \in \mathbb{N} \Rightarrow n \geq 0).

      Common Structures

      Universal Implication

      x.(P(x)Q(x))\forall x. (P(x) \Rightarrow Q(x)) means “for all xx, if P(x)P(x) then Q(x)Q(x)”.

      To prove:

      1. Fix arbitrary xx
      2. Assume P(x)P(x)
      3. Prove Q(x)Q(x)

      Hidden Quantifiers

      Many statements hide quantifiers:

      “The product of two odd integers is odd.”

      Formally: m,nZ.(Odd(m)Odd(n))Odd(mn)\forall m, n \in \mathbb{Z}. (\text{Odd}(m) \land \text{Odd}(n)) \Rightarrow \text{Odd}(m \cdot n)

      Common Pitfalls

      Forgetting to Fix Arbitrarily

      Trap: Proving for a specific value does not prove “for all”.

      Proving P(0)P(0), P(1)P(1), P(2)P(2) does not prove n.P(n)\forall n. P(n).

      Using Non-arbitrary Values

      Trap: If you pick x=5x = 5 and prove P(5)P(5), you have not proved x.P(x)\forall x. P(x).

      You must prove it for an arbitrary, unspecified xx.

      Assuming the Consequent

      Trap: When proving x.(P(x)Q(x))\forall x. (P(x) \Rightarrow Q(x)), you assume P(x)P(x), not Q(x)Q(x).


      Summary

      • x.P(x)\forall x. P(x) means ”P(x)P(x) is true for all xx
      • Proof by universal generalisation: fix arbitrary xx, prove P(x)P(x)
      • x.y.P(x,y)y.x.P(x,y)\forall x. \forall y. P(x, y) \equiv \forall y. \forall x. P(x, y)
      • Hidden quantifiers must be made explicit for rigorous proof
    • Existential Quantification

      The Existential Quantifier

      The existential quantifier \exists means “there exists” or “there is”.

      The statement x.P(x)\exists x. P(x) means: there is at least one element xx (in the domain) such that P(x)P(x) holds.

      Syntax

      xS.P(x)\exists x \in S. P(x)

      means “there exists an xx in the set SS such that P(x)P(x) holds.”

      Examples

      • nN.n>10\exists n \in \mathbb{N}. n > 10 (true: n=11n = 11 works)
      • nZ.n2=4\exists n \in \mathbb{Z}. n^2 = 4 (true: n=2n = 2 or n=2n = -2)
      • nZ.n2=2\exists n \in \mathbb{Z}. n^2 = 2 (false: no integer squared equals 2)

      Proof Strategy: Constructive Existence

      To prove xS.P(x)\exists x \in S. P(x):

      1. Exhibit a witness: Find a specific value x0Sx_0 \in S
      2. Verify: Prove that P(x0)P(x_0) holds

      This is called a constructive proof of existence.

      Example

      Statement: There exists an even prime number.

      Proof:

      Take x0=2x_0 = 2.

      We verify: 22 is even (since 2=212 = 2 \cdot 1) and 22 is prime (its only divisors are 11 and 22).

      Therefore, nN\exists n \in \mathbb{N} such that nn is even and prime.

      Non-constructive Existence

      Sometimes we can prove x.P(x)\exists x. P(x) without exhibiting a specific witness.

      Example: “There exist irrational numbers a,ba, b such that aba^b is rational.”

      Non-constructive proof: Consider 22\sqrt{2}^{\sqrt{2}}.

      • If it’s rational, take a=2a = \sqrt{2}, b=2b = \sqrt{2}
      • If it’s irrational, take a=22a = \sqrt{2}^{\sqrt{2}}, b=2b = \sqrt{2}, and note ab=(22)2=22=2a^b = (\sqrt{2}^{\sqrt{2}})^{\sqrt{2}} = \sqrt{2}^2 = 2

      In either case, such a,ba, b exist.

      Nested Quantifiers

      x.y.P(x,y)\exists x. \forall y. P(x, y)

      “There exists an xx such that for all yy, P(x,y)P(x, y) holds.”

      y.x.P(x,y)\forall y. \exists x. P(x, y)

      “For all yy, there exists an xx such that P(x,y)P(x, y) holds.”

      Order Matters!

      These are NOT equivalent:

      x.y.P(x,y)≢y.x.P(x,y)\exists x. \forall y. P(x, y) \not\equiv \forall y. \exists x. P(x, y)

      Example

      Consider P(x,y):P(x, y) : \equivxx loves yy” over people.

      • x.y.P(x,y)\exists x. \forall y. P(x, y): There is someone who loves everyone.
      • y.x.P(x,y)\forall y. \exists x. P(x, y): Everyone is loved by someone.

      The second is much weaker.

      Example over R\mathbb{R}

      • x.y.xy\exists x. \forall y. x \leq y: False (no smallest real)
      • y.x.xy\forall y. \exists x. x \leq y: True (take x=y1x = y - 1)

      De Morgan’s Laws for Quantifiers

      Negation of Universal

      ¬(x.P(x))x.¬P(x)\neg(\forall x. P(x)) \equiv \exists x. \neg P(x)

      “It is not the case that all xx satisfy PP” means “there exists an xx that does not satisfy PP.”

      Negation of Existential

      ¬(x.P(x))x.¬P(x)\neg(\exists x. P(x)) \equiv \forall x. \neg P(x)

      “It is not the case that some xx satisfes PP” means “all xx fail to satisfy PP.”

      Example

      ¬(nN.n>0)nN.n0\neg(\forall n \in \mathbb{N}. n > 0) \equiv \exists n \in \mathbb{N}. n \leq 0

      The witness is n=0n = 0.

      Common Structures

      Existential Implication

      x.(P(x)Q(x))\exists x. (P(x) \Rightarrow Q(x)) is tricky.

      Note: P(x)Q(x)P(x) \Rightarrow Q(x) is equivalent to ¬P(x)Q(x)\neg P(x) \lor Q(x).

      So x.(P(x)Q(x))\exists x. (P(x) \Rightarrow Q(x)) is equivalent to x.(¬P(x)Q(x))\exists x. (\neg P(x) \lor Q(x)).

      Example

      xR.(x<0x2<0)\exists x \in \mathbb{R}. (x < 0 \Rightarrow x^2 < 0)

      This is true: take x=1x = 1. Then x<0x < 0 is false, so the implication is vacuously true.

      Common Pitfalls

      Unverified Witness

      Trap: Claiming x.P(x)\exists x. P(x) without verifying P(x)P(x) for your candidate.

      Example: To prove n.n2+n+41\exists n. n^2 + n + 41 is prime, you cannot just say “take n=40n = 40” without checking.

      Wrong Quantifier Order

      Trap: Confusing x.y\exists x. \forall y with y.x\forall y. \exists x.

      They are NOT equivalent.

      Using a Variable Without Introduction

      Trap: Using xx in a proof without specifying where it comes from.

      When proving x.y.P(x,y)\forall x. \exists y. P(x, y), for each xx you must say how to find yy.


      Summary

      • x.P(x)\exists x. P(x) means “there exists an xx such that P(x)P(x)
      • Prove constructively: exhibit a witness x0x_0 and verify P(x0)P(x_0)
      • Quantifier order matters: x.y≢y.x\exists x. \forall y \not\equiv \forall y. \exists x
      • De Morgan: ¬x.P(x)x.¬P(x)\neg\forall x. P(x) \equiv \exists x. \neg P(x) and ¬x.P(x)x.¬P(x)\neg\exists x. P(x) \equiv \forall x. \neg P(x)
    • Conjunction and Disjunction

      Conjunction (And)

      The conjunction PQP \land Q means ”PP and QQ”.

      Truth Table

      PPQQPQP \land Q
      TTT
      TFF
      FTF
      FFF

      PQP \land Q is true only when both PP and QQ are true.

      Proof Strategy

      To prove PQP \land Q:

      1. Prove PP
      2. Prove QQ

      Both must be proved separately.

      Disjunction (Or)

      The disjunction PQP \lor Q means ”PP or QQ”.

      In mathematics, “or” is inclusive: PQP \lor Q is true if at least one of PP, QQ is true.

      Truth Table

      PPQQPQP \lor Q
      TTT
      TFT
      FTT
      FFF

      Proof Strategies for Disjunction

      Method 1: Prove One Disjunct

      Prove PP (then PQP \lor Q follows) or prove QQ.

      Method 2: Proof by Cases (Disjunction Elimination)

      To prove RR from PQP \lor Q:

      1. Case 1: Assume PP, prove RR
      2. Case 2: Assume QQ, prove RR
      3. Since PQP \lor Q and RR follows in either case, conclude RR

      Method 3: Proof by Contradiction

      To prove PQP \lor Q:

      1. Assume ¬(PQ)\neg(P \lor Q)
      2. By De Morgan, this means ¬P¬Q\neg P \land \neg Q
      3. Derive a contradiction

      Example: Proof by Cases

      Statement: For all integers nn, n2nn^2 \geq n.

      This is false. But let us prove: For all integers nn, n20n^2 \geq 0.

      Proof by cases on n0n \geq 0 or n<0n < 0:

      Case 1: n0n \geq 0.

      Then n20n=0n^2 \geq 0 \cdot n = 0.

      Case 2: n<0n < 0.

      Then n=kn = -k for some k>0k > 0. So n2=(k)2=k2>0n^2 = (-k)^2 = k^2 > 0.

      In either case, n20n^2 \geq 0.

      De Morgan’s Laws

      For Statements

      ¬(PQ)¬P¬Q\neg(P \land Q) \equiv \neg P \lor \neg Q ¬(PQ)¬P¬Q\neg(P \lor Q) \equiv \neg P \land \neg Q

      For Sets

      S(AB)=(SA)(SB)S \setminus (A \cap B) = (S \setminus A) \cup (S \setminus B) S(AB)=(SA)(SB)S \setminus (A \cup B) = (S \setminus A) \cap (S \setminus B)

      Verification by Truth Table

      PPQQ¬(PQ)\neg(P \land Q)¬P¬Q\neg P \lor \neg Q
      TTFF
      TFTT
      FTTT
      FFTT

      Distributivity

      Conjunction over Disjunction

      P(QR)(PQ)(PR)P \land (Q \lor R) \equiv (P \land Q) \lor (P \land R)

      Disjunction over Conjunction

      P(QR)(PQ)(PR)P \lor (Q \land R) \equiv (P \lor Q) \land (P \lor R)

      Associativity and Commutativity

      Both \land and \lor are:

      • Associative: P(QR)(PQ)RP \land (Q \land R) \equiv (P \land Q) \land R
      • Commutative: PQQPP \land Q \equiv Q \land P

      This allows us to write P1P2PnP_1 \land P_2 \land \cdots \land P_n and P1P2PnP_1 \lor P_2 \lor \cdots \lor P_n without parentheses.

      Absorption Laws

      P(PQ)PP \land (P \lor Q) \equiv P P(PQ)PP \lor (P \land Q) \equiv P

      Idempotent Laws

      PPPP \land P \equiv P PPPP \lor P \equiv P

      Identity Laws

      PTruePP \land \text{True} \equiv P PFalsePP \lor \text{False} \equiv P

      Domination Laws

      PFalseFalseP \land \text{False} \equiv \text{False} PTrueTrueP \lor \text{True} \equiv \text{True}

      Using Logical Equivalences

      These equivalences allow us to transform statements into simpler or more convenient forms.

      Example

      Simplify ¬(PQ)P\neg(P \land Q) \land P.

      ¬(PQ)P(¬P¬Q)P\neg(P \land Q) \land P \equiv (\neg P \lor \neg Q) \land P (De Morgan) (¬PP)(¬QP)\equiv (\neg P \land P) \lor (\neg Q \land P) (Distributivity) False(¬QP)\equiv \text{False} \lor (\neg Q \land P) (Contradiction) ¬QP\equiv \neg Q \land P (Identity)

      Common Pitfalls

      Confusing And/Or

      Trap: “P or Q” in mathematics includes the case where both are true.

      This is different from everyday English where “or” often means “exactly one”.

      Forgetting a Case

      Trap: When using proof by cases, check you have covered all possibilities.

      If your cases are n>0n > 0 and n<0n < 0, you forgot n=0n = 0.

      Overlapping Cases

      Trap: Cases need not be disjoint, but each case must be proved.

      Cases may overlap; what matters is each case individually leads to the conclusion.


      Summary

      • PQP \land Q requires proving both PP and QQ
      • PQP \lor Q can be proved by proving one, or by cases
      • De Morgan’s laws: ¬(PQ)¬P¬Q\neg(P \land Q) \equiv \neg P \lor \neg Q and ¬(PQ)¬P¬Q\neg(P \lor Q) \equiv \neg P \land \neg Q
      • Both connectives are associative, commutative, and distributive over each other
    • Negation and Proof by Contradiction

      Negation

      The negation of a statement PP, written ¬P\neg P, is true when PP is false and false when PP is true.

      Truth Table

      PP¬P\neg P
      TF
      FT

      Double Negation

      ¬(¬P)P\neg(\neg P) \equiv P

      Negation of Compound Statements

      Negation of Conjunction and Disjunction

      ¬(PQ)¬P¬Q\neg(P \land Q) \equiv \neg P \lor \neg Q ¬(PQ)¬P¬Q\neg(P \lor Q) \equiv \neg P \land \neg Q

      Negation of Implication

      ¬(PQ)P¬Q\neg(P \Rightarrow Q) \equiv P \land \neg Q

      This is because PQ¬PQP \Rightarrow Q \equiv \neg P \lor Q, so:

      ¬(PQ)¬(¬PQ)P¬Q\neg(P \Rightarrow Q) \equiv \neg(\neg P \lor Q) \equiv P \land \neg Q

      Negation of Quantifiers

      ¬(x.P(x))x.¬P(x)\neg(\forall x. P(x)) \equiv \exists x. \neg P(x) ¬(x.P(x))x.¬P(x)\neg(\exists x. P(x)) \equiv \forall x. \neg P(x)

      Negation of Bi-implication

      ¬(PQ)(P¬Q)(¬PQ)\neg(P \Leftrightarrow Q) \equiv (P \land \neg Q) \lor (\neg P \land Q)

      The negation says “exactly one of PP, QQ holds.”

      Proof by Contradiction

      The Principle

      To prove PP by contradiction:

      1. Assume ¬P\neg P
      2. Derive a contradiction (e.g., Q¬QQ \land \neg Q for some statement QQ)
      3. Conclude PP must be true

      This is also called reductio ad absurdum.

      Logical Justification

      From ¬PFalse\neg P \Rightarrow \text{False}, we conclude PP.

      Since ¬PFalse\neg P \Rightarrow \text{False} is equivalent to ¬(¬P)FalsePFalseP\neg(\neg P) \lor \text{False} \equiv P \lor \text{False} \equiv P.

      Examinable Proof: Irrationality of 2\sqrt{2}

      Theorem: 2\sqrt{2} is irrational.

      Proof by contradiction:

      Suppose 2\sqrt{2} is rational. Then 2=pq\sqrt{2} = \frac{p}{q} for integers p,qp, q with gcd(p,q)=1\gcd(p, q) = 1 (the fraction is in lowest terms).

      Squaring: 2=p2q22 = \frac{p^2}{q^2}, so p2=2q2p^2 = 2q^2.

      This means p2p^2 is even. Since nn even implies n2n^2 even, and n2n^2 even implies nn even (contrapositive), we have pp is even.

      Write p=2kp = 2k for some integer kk.

      Substituting: 4k2=2q24k^2 = 2q^2, so q2=2k2q^2 = 2k^2.

      Thus q2q^2 is even, so qq is even.

      But pp and qq are both even, contradicting gcd(p,q)=1\gcd(p, q) = 1.

      Therefore 2\sqrt{2} is irrational.

      Examinable Proof: Infinitude of Primes

      Theorem: There are infinitely many prime numbers.

      Proof by contradiction (Euclid):

      Suppose there are finitely many primes: p1,p2,,pnp_1, p_2, \ldots, p_n.

      Let N=p1p2pn+1N = p_1 \cdot p_2 \cdot \cdots \cdot p_n + 1.

      Since N>1N > 1, the Fundamental Theorem of Arithmetic says NN has a prime divisor pp.

      If p=pip = p_i for some ii, then pp1p2pnp \mid p_1 \cdot p_2 \cdot \cdots \cdot p_n and pNp \mid N.

      This implies p(Np1p2pn)=1p \mid (N - p_1 \cdot p_2 \cdot \cdots \cdot p_n) = 1.

      But no prime divides 1. Contradiction.

      Therefore pp is not in our list, so there is a prime not among p1,,pnp_1, \ldots, p_n.

      This contradicts the assumption of finitely many primes.

      Therefore there are infinitely many primes.

      When to Use Contradiction

      Contradiction is useful when:

      1. The negation gives better structure (e.g., proving irrationals)
      2. Direct proof seems impossible or awkward
      3. The statement asserts non-existence or impossibility

      Example: Negation Simplifies Structure

      Statement: There are no integers a,ba, b such that a2=2b2a^2 = 2b^2.

      If we negate: “There exist integers a,ba, b such that a2=2b2a^2 = 2b^2.”

      This gives us something to work with algebraically.

      Contradiction vs Contrapositive

      If you find yourself assuming ¬P\neg P and deriving ¬Q\neg Q, you might instead prove the contrapositive QPQ \Rightarrow P directly.

      Both are valid; contrapositive is often clearer when applicable.

      Example

      Statement: If n2n^2 is even, then nn is even.

      • Contrapositive proof: If nn is odd, then n2n^2 is odd. (Direct)
      • Contradiction proof: Suppose n2n^2 is even and nn is odd. Then n2n^2 is odd. Contradiction.

      Both work; contrapositive is simpler here.

      Common Pitfalls

      Forgetting to Negate Assumptions

      Trap: In proof by contradiction, you assume ¬P\neg P. Some students assume PP instead.

      Circular Reasoning

      Trap: Assuming what you want to prove within the proof.

      This defeats the purpose of proof by contradiction.

      Overusing Contradiction

      Trap: Using contradiction when a direct proof is simpler.

      Direct proofs are often clearer. Use contradiction when necessary.


      Summary

      • ¬(PQ)¬P¬Q\neg(P \land Q) \equiv \neg P \lor \neg Q and ¬(PQ)¬P¬Q\neg(P \lor Q) \equiv \neg P \land \neg Q
      • ¬(PQ)P¬Q\neg(P \Rightarrow Q) \equiv P \land \neg Q
      • Proof by contradiction: assume ¬P\neg P, derive False\text{False}
      • Examinable proofs: irrationality of 2\sqrt{2}, infinitude of primes
    • Divisibility and the Division Theorem

      Divisibility

      Definition

      For integers aa and bb, we say aa divides bb, written aba \mid b, if and only if there exists an integer kk such that b=akb = ak.

      ab    kZ.b=aka \mid b \iff \exists k \in \mathbb{Z}. b = ak

      Terminology

      • aa is a divisor or factor of bb
      • bb is a multiple of aa
      • If aba \nmid b, then aa does not divide bb

      Examples

      • 3123 \mid 12 since 12=3412 = 3 \cdot 4
      • 312-3 \mid 12 since 12=(3)(4)12 = (-3) \cdot (-4)
      • 5125 \nmid 12 since no integer kk satisfies 12=5k12 = 5k
      • a0a \mid 0 for all a0a \neq 0 since 0=a00 = a \cdot 0

      Properties of Divisibility

      Reflexivity

      aafor all aZa \mid a \quad \text{for all } a \in \mathbb{Z}

      Transitivity

      abbcaca \mid b \land b \mid c \Rightarrow a \mid c

      Proof: If b=akb = ak and c=bc = b\ell, then c=ak=a(k)c = ak\ell = a(k\ell).

      Linearity

      abaca(bx+cy) for all x,yZa \mid b \land a \mid c \Rightarrow a \mid (bx + cy) \text{ for all } x, y \in \mathbb{Z}

      Proof: If b=akb = ak and c=ac = a\ell, then bx+cy=akx+ay=a(kx+y)bx + cy = akx + a\ell y = a(kx + \ell y).

      Antisymmetry (up to sign)

      abbaa=±ba \mid b \land b \mid a \Rightarrow a = \pm b

      Proof: If aba \mid b and bab \mid a, then b=akb = ak and a=ba = b\ell for integers k,k, \ell.

      Hence a=aka = ak\ell, so a(1k)=0a(1 - k\ell) = 0.

      If a=0a = 0, then b=0b = 0 and a=ba = b.

      If a0a \neq 0, then k=1k\ell = 1. Since k,Zk, \ell \in \mathbb{Z}, we have (k,)=(1,1)(k, \ell) = (1, 1) or (1,1)(-1, -1).

      Thus b=ab = a or b=ab = -a.

      The Division Theorem

      Statement

      Theorem (Division Theorem): For any integer aa and positive integer b>0b > 0, there exist unique integers qq and rr such that:

      a=bq+rwith 0r<ba = bq + r \quad \text{with } 0 \leq r < b

      • qq is the quotient (how many times bb fits into aa)
      • rr is the remainder (what is left over)
      • We write r=amodbr = a \bmod b

      Examples

      aabbqqrrVerification
      1753217=53+217 = 5 \cdot 3 + 2
      -175-4317=5(4)+3-17 = 5 \cdot (-4) + 3
      2072620=72+620 = 7 \cdot 2 + 6
      1427014=27+014 = 2 \cdot 7 + 0

      Proof Outline

      Existence:

      Consider the set S={abk:kZ,abk0}S = \{a - bk : k \in \mathbb{Z}, a - bk \geq 0\}.

      This set is non-empty (take kk large and negative if a<0a < 0, or k=0k = 0 if a0a \geq 0).

      Let rr be the smallest element of SS. Then r=abq0r = a - bq \geq 0 for some qq.

      If rbr \geq b, then rb=ab(q+1)0r - b = a - b(q+1) \geq 0, contradicting minimality.

      So 0r<b0 \leq r < b.

      Uniqueness:

      If a=bq1+r1=bq2+r2a = bq_1 + r_1 = bq_2 + r_2 with 0r1,r2<b0 \leq r_1, r_2 < b:

      Then b(q1q2)=r2r1b(q_1 - q_2) = r_2 - r_1.

      We have r2r1<b|r_2 - r_1| < b, so b(r2r1)b \mid (r_2 - r_1) forces r2r1=0r_2 - r_1 = 0.

      Thus r1=r2r_1 = r_2 and hence q1=q2q_1 = q_2.

      Computing Quotient and Remainder

      For Non-negative aa

      q=a/bandr=abqq = \lfloor a/b \rfloor \quad \text{and} \quad r = a - bq

      For Negative aa

      Be careful: 17mod5=3-17 \bmod 5 = 3, not 2-2.

      The standard convention requires 0r<b0 \leq r < b:

      r=ababr = a - b \cdot \left\lfloor \frac{a}{b} \right\rfloor

      Properties of Remainder

      Invariance under Addition

      If ar(modb)a \equiv r \pmod{b} and cs(modb)c \equiv s \pmod{b} with 0r,s<b0 \leq r, s < b, then:

      a+cr+s(modb)a + c \equiv r + s \pmod{b}

      Invariance under Multiplication

      If ar(modb)a \equiv r \pmod{b} and cs(modb)c \equiv s \pmod{b} with 0r,s<b0 \leq r, s < b, then:

      acrs(modb)ac \equiv rs \pmod{b}


      Summary

      • aba \mid b means there exists kk with b=akb = ak
      • Divisibility is reflexive, transitive, and antisymmetric up to sign
      • The Division Theorem: for b>0b > 0, unique q,rq, r with a=bq+ra = bq + r and 0r<b0 \leq r < b
      • Remainder satisfies 0r<b0 \leq r < b even for negative aa
    • Modular Arithmetic

      Congruence Modulo nn

      Definition

      For integers a,ba, b and a positive integer n1n \geq 1, we say aa is congruent to bb modulo nn, written ab(modn)a \equiv b \pmod{n}, if and only if:

      n(ab)n \mid (a - b)

      Equivalently, ab(modn)a \equiv b \pmod{n} iff aa and bb have the same remainder when divided by nn.

      Examples

      • 175(mod12)17 \equiv 5 \pmod{12} since 175=1217 - 5 = 12 and 121212 \mid 12
      • 232(mod7)23 \equiv 2 \pmod{7} since 232=2123 - 2 = 21 and 7217 \mid 21
      • 53(mod8)-5 \equiv 3 \pmod{8} since 53=8-5 - 3 = -8 and 888 \mid -8

      Congruence is an Equivalence Relation

      Reflexivity

      aa(modn)a \equiv a \pmod{n}

      Since n(aa)=0n \mid (a - a) = 0 for all nn.

      Symmetry

      ab(modn)ba(modn)a \equiv b \pmod{n} \Rightarrow b \equiv a \pmod{n}

      Since n(ab)n \mid (a - b) implies n(ba)n \mid (b - a).

      Transitivity

      ab(modn)bc(modn)ac(modn)a \equiv b \pmod{n} \land b \equiv c \pmod{n} \Rightarrow a \equiv c \pmod{n}

      Proof: n(ab)n \mid (a - b) and n(bc)n \mid (b - c) implies n[(ab)+(bc)]=acn \mid [(a-b) + (b-c)] = a - c.

      Congruence Classes

      The congruence class of aa modulo nn is:

      [a]n={bZ:ab(modn)}={a+kn:kZ}[a]_n = \{b \in \mathbb{Z} : a \equiv b \pmod{n}\} = \{a + kn : k \in \mathbb{Z}\}

      Examples

      • [0]3={,6,3,0,3,6,9,}[0]_3 = \{\ldots, -6, -3, 0, 3, 6, 9, \ldots\}
      • [1]3={,5,2,1,4,7,10,}[1]_3 = \{\ldots, -5, -2, 1, 4, 7, 10, \ldots\}
      • [2]3={,4,1,2,5,8,11,}[2]_3 = \{\ldots, -4, -1, 2, 5, 8, 11, \ldots\}

      Number of Classes

      There are exactly nn congruence classes modulo nn: [0]n,[1]n,,[n1]n[0]_n, [1]_n, \ldots, [n-1]_n.

      The standard representatives are Zn={0,1,2,,n1}\mathbb{Z}_n = \{0, 1, 2, \ldots, n-1\}.

      Arithmetic Operations on Congruences

      Addition

      If ab(modn)a \equiv b \pmod{n} and cd(modn)c \equiv d \pmod{n}, then:

      a+cb+d(modn)a + c \equiv b + d \pmod{n}

      Multiplication

      If ab(modn)a \equiv b \pmod{n} and cd(modn)c \equiv d \pmod{n}, then:

      acbd(modn)ac \equiv bd \pmod{n}

      Powers

      If ab(modn)a \equiv b \pmod{n}, then for any k0k \geq 0:

      akbk(modn)a^k \equiv b^k \pmod{n}

      Proof (Multiplication)

      Given ab(modn)a \equiv b \pmod{n} and cd(modn)c \equiv d \pmod{n}:

      acbd=acbc+bcbd=c(ab)+b(cd)ac - bd = ac - bc + bc - bd = c(a - b) + b(c - d)

      Since n(ab)n \mid (a - b) and n(cd)n \mid (c - d), we have n[c(ab)+b(cd)]=acbdn \mid [c(a-b) + b(c-d)] = ac - bd.

      Thus acbd(modn)ac \equiv bd \pmod{n}.

      Common Pitfall: Cancellation

      The Problem

      acbc(modn)⇏ab(modn)ac \equiv bc \pmod{n} \not\Rightarrow a \equiv b \pmod{n}

      Counterexample

      2320(mod6)2 \cdot 3 \equiv 2 \cdot 0 \pmod{6} but 3≢0(mod6)3 \not\equiv 0 \pmod{6}.

      When Does Cancellation Work?

      If gcd(c,n)=1\gcd(c, n) = 1 and acbc(modn)ac \equiv bc \pmod{n}, then ab(modn)a \equiv b \pmod{n}.

      Proof: acbc(modn)ac \equiv bc \pmod{n} means nc(ab)n \mid c(a - b).

      Since gcd(c,n)=1\gcd(c, n) = 1, Euclid’s lemma gives n(ab)n \mid (a - b).

      Thus ab(modn)a \equiv b \pmod{n}.

      Divisibility Tests

      Test for 2

      akak1a1a0|\overline{a_k a_{k-1} \ldots a_1 a_0}| is divisible by 2 iff a0a_0 (the last digit) is even.

      Reason: 100(mod2)10 \equiv 0 \pmod{2}, so ak10k++a110+a0a0(mod2)a_k \cdot 10^k + \cdots + a_1 \cdot 10 + a_0 \equiv a_0 \pmod{2}.

      Test for 3

      A number is divisible by 3 iff the sum of its digits is divisible by 3.

      Reason: 101(mod3)10 \equiv 1 \pmod{3}, so ak10k++a110+a0ak++a0(mod3)a_k \cdot 10^k + \cdots + a_1 \cdot 10 + a_0 \equiv a_k + \cdots + a_0 \pmod{3}.

      Test for 9

      A number is divisible by 9 iff the sum of its digits is divisible by 9.

      Reason: 101(mod9)10 \equiv 1 \pmod{9}.

      Test for 11

      A number is divisible by 11 iff the alternating sum of digits (a0a1+a2)(a_0 - a_1 + a_2 - \cdots) is divisible by 11.

      Reason: 101(mod11)10 \equiv -1 \pmod{11}.


      Summary

      • ab(modn)a \equiv b \pmod{n} iff n(ab)n \mid (a - b)
      • Congruence is an equivalence relation
      • Congruences preserve addition and multiplication
      • Cancellation requires gcd(c,n)=1\gcd(c, n) = 1
      • Divisibility tests use modular arithmetic
    • Greatest Common Divisor and Euclid's Algorithm

      Greatest Common Divisor

      Definition

      For integers aa and bb (not both zero), the greatest common divisor gcd(a,b)\gcd(a, b) is the largest positive integer dd such that dad \mid a and dbd \mid b.

      By convention, gcd(0,0)=0\gcd(0, 0) = 0.

      Examples

      • gcd(12,18)=6\gcd(12, 18) = 6
      • gcd(17,5)=1\gcd(17, 5) = 1
      • gcd(100,0)=100\gcd(100, 0) = 100

      Coprimality

      We say aa and bb are coprime or relatively prime if gcd(a,b)=1\gcd(a, b) = 1.

      Properties of GCD

      Symmetry

      gcd(a,b)=gcd(b,a)\gcd(a, b) = \gcd(b, a)

      Absolute Value

      gcd(a,b)=gcd(a,b)\gcd(a, b) = \gcd(|a|, |b|)

      If aba \mid b

      If a0a \neq 0 and aba \mid b, then gcd(a,b)=a\gcd(a, b) = |a|.

      Adding Multiples

      For any integers k,k, \ell:

      gcd(a,b)=gcd(akb,b)\gcd(a, b) = \gcd(a - kb, b)

      This is the key insight behind Euclid’s algorithm.

      Euclid’s Algorithm

      The Key Lemma

      gcd(a,b)=gcd(b,amodb)\gcd(a, b) = \gcd(b, a \bmod b)

      Proof: Let a=bq+ra = bq + r with 0r<b0 \leq r < b.

      Any common divisor of aa and bb divides r=abqr = a - bq.

      Any common divisor of bb and rr divides a=bq+ra = bq + r.

      Thus {d:dadb}={d:dbdr}\{d : d \mid a \land d \mid b\} = \{d : d \mid b \land d \mid r\}.

      Taking the maximum, gcd(a,b)=gcd(b,r)=gcd(b,amodb)\gcd(a, b) = \gcd(b, r) = \gcd(b, a \bmod b).

      The Algorithm

      Euclid’s algorithm computes gcd(a,b)\gcd(a, b) recursively:

      function gcd(a, b):
          if b = 0:
              return a
          else:
              return gcd(b, a mod b)

      Termination

      The second argument strictly decreases at each step: amodb<ba \bmod b < b.

      Since b0b \geq 0 and the non-negative integers are well-ordered, the algorithm terminates.

      Example

      Compute gcd(48,18)\gcd(48, 18):

      Stepaabbamodba \bmod b
      1481812
      218126
      31260
      460

      gcd(48,18)=6\gcd(48, 18) = 6.

      Bézout’s Identity

      Statement

      Theorem (Bézout’s Identity): For integers aa and bb (not both zero), there exist integers xx and yy such that:

      ax+by=gcd(a,b)ax + by = \gcd(a, b)

      Extended Euclidean Algorithm

      We compute xx and yy alongside the GCD by tracking linear combinations.

      Example: Find integers x,yx, y with 48x+18y=gcd(48,18)=648x + 18y = \gcd(48, 18) = 6.

      Working through Euclid’s algorithm:

      • 48=218+1248 = 2 \cdot 18 + 12
      • 18=112+618 = 1 \cdot 12 + 6
      • 12=26+012 = 2 \cdot 6 + 0

      Back-substituting:

      • 6=181126 = 18 - 1 \cdot 12
      • 6=181(48218)=3181486 = 18 - 1 \cdot (48 - 2 \cdot 18) = 3 \cdot 18 - 1 \cdot 48

      So x=1x = -1 and y=3y = 3 verify 48(1)+183=48+54=648 \cdot (-1) + 18 \cdot 3 = -48 + 54 = 6.

      Euclid’s Lemma

      Statement

      Lemma (Euclid’s Lemma): If pp is prime and pabp \mid ab, then pap \mid a or pbp \mid b.

      Proof

      Suppose pap \nmid a. Since pp is prime, gcd(p,a)=1\gcd(p, a) = 1.

      By Bézout’s Identity, there exist x,yx, y with px+ay=1px + ay = 1.

      Multiply by bb: pxb+aby=bpxb + aby = b.

      Since pabp \mid ab, we have pabyp \mid aby. Clearly ppxbp \mid pxb.

      Thus p(pxb+aby)=bp \mid (pxb + aby) = b.

      Multiplicativity of GCD

      Statement

      If gcd(b,c)=1\gcd(b, c) = 1, then:

      gcd(a,bc)=gcd(a,b)gcd(a,c)\gcd(a, bc) = \gcd(a, b) \cdot \gcd(a, c)

      Proof Sketch

      Let d1=gcd(a,b)d_1 = \gcd(a, b) and d2=gcd(a,c)d_2 = \gcd(a, c).

      Since gcd(b,c)=1\gcd(b, c) = 1 and d1bd_1 \mid b, d2cd_2 \mid c, we have gcd(d1,d2)=1\gcd(d_1, d_2) = 1.

      Both d1d_1 and d2d_2 divide aa, so d1d2d_1 d_2 divides aa.

      Both d1d_1 and d2d_2 divide bcbc (since d1bd_1 \mid b and d2cd_2 \mid c), so d1d2d_1 d_2 divides bcbc.

      Thus d1d2gcd(a,bc)d_1 d_2 \mid \gcd(a, bc).

      For the reverse, use Bézout’s identity with the coprimality assumptions.


      Summary

      • gcd(a,b)\gcd(a, b) is the largest common divisor, defined for a,ba, b not both zero
      • Key lemma: gcd(a,b)=gcd(b,amodb)\gcd(a, b) = \gcd(b, a \bmod b)
      • Euclid’s algorithm terminates by well-foundedness
      • Bézout’s Identity: ax+by=gcd(a,b)ax + by = \gcd(a, b) for some x,yx, y
      • Euclid’s Lemma: if prime pabp \mid ab, then pap \mid a or pbp \mid b
    • Fermat's Little Theorem

      Statement of Fermat’s Little Theorem

      Theorem

      Fermat’s Little Theorem: If pp is a prime and aa is an integer not divisible by pp, then:

      ap11(modp)a^{p-1} \equiv 1 \pmod{p}

      Equivalent Formulation

      For any integer aa and prime pp:

      apa(modp)a^p \equiv a \pmod{p}

      This form works even when pap \mid a (both sides are 0modp0 \bmod p).

      Examples

      • p=5p = 5, a=2a = 2: 24=161(mod5)2^4 = 16 \equiv 1 \pmod{5}
      • p=7p = 7, a=3a = 3: 36=7291(mod7)3^6 = 729 \equiv 1 \pmod{7}
      • p=11p = 11, a=2a = 2: 210=10241(mod11)2^{10} = 1024 \equiv 1 \pmod{11}

      Examinable Proof

      The Proof Strategy

      We show that a,2a,3a,,(p1)aa, 2a, 3a, \ldots, (p-1)a modulo pp form a permutation of 1,2,3,,p11, 2, 3, \ldots, p-1.

      Step 1: Non-zeroness

      Each ia(modp)ia \pmod{p} is non-zero for 1ip11 \leq i \leq p-1.

      Reason: pap \nmid a by hypothesis, and pip \nmid i since 1i<p1 \leq i < p and pp is prime.

      Thus piap \nmid ia.

      Step 2: Distinctness

      If iaja(modp)ia \equiv ja \pmod{p} for 1i,jp11 \leq i, j \leq p-1, then i=ji = j.

      Proof: iaja(modp)ia \equiv ja \pmod{p} means p(ij)ap \mid (i-j)a.

      Since pap \nmid a and pp is prime, Euclid’s lemma gives p(ij)p \mid (i-j).

      But ijp2<p|i - j| \leq p - 2 < p, so ij=0|i - j| = 0 and i=ji = j.

      Step 3: Permutation Argument

      The numbers 1,2,,p11, 2, \ldots, p-1 are p1p-1 distinct non-zero residues modulo pp.

      By Steps 1 and 2, a,2a,,(p1)aa, 2a, \ldots, (p-1)a are also p1p-1 distinct non-zero residues modulo pp.

      Therefore:

      {a,2a,,(p1)a}={1,2,,p1}(modp)\{a, 2a, \ldots, (p-1)a\} = \{1, 2, \ldots, p-1\} \pmod{p}

      Step 4: Product Equality

      Taking products:

      a2a3a(p1)a123(p1)(modp)a \cdot 2a \cdot 3a \cdot \cdots \cdot (p-1)a \equiv 1 \cdot 2 \cdot 3 \cdot \cdots \cdot (p-1) \pmod{p}

      This simplifies to:

      ap1(p1)!(p1)!(modp)a^{p-1} \cdot (p-1)! \equiv (p-1)! \pmod{p}

      Step 5: Cancellation

      Since 1(p1)!<p!1 \leq (p-1)! < p! and (p1)!(p-1)! is the product of integers 11 through p1p-1, none of which are divisible by pp, we have p(p1)!p \nmid (p-1)!.

      Thus gcd(p,(p1)!)=1\gcd(p, (p-1)!) = 1.

      By the cancellation property of modular arithmetic:

      ap11(modp)a^{p-1} \equiv 1 \pmod{p}

      Applications

      Modular Exponentiation

      To compute akmodpa^k \bmod p where pp is prime:

      1. Reduce kk modulo p1p-1: k=q(p1)+rk = q(p-1) + r with 0r<p10 \leq r < p-1
      2. Then akar(modp)a^k \equiv a^r \pmod{p}

      Example: Compute 3100mod73^{100} \bmod 7.

      100=166+4100 = 16 \cdot 6 + 4, so 310034=814(mod7)3^{100} \equiv 3^4 = 81 \equiv 4 \pmod{7}.

      Primality Testing

      If there exists aa with pap \nmid a such that ap1≢1(modp)a^{p-1} \not\equiv 1 \pmod{p}, then pp is not prime.

      Note: The converse is not true in general. A Carmichael number is composite but satisfies an11(modn)a^{n-1} \equiv 1 \pmod{n} for all aa coprime to nn.

      Diffie-Hellman Key Exchange

      Two parties agree on:

      • A large prime pp
      • A generator gg of (Z/pZ)×(\mathbb{Z}/p\mathbb{Z})^\times

      Each picks a secret:

      • Alice picks secret aa, sends gamodpg^a \bmod p
      • Bob picks secret bb, sends gbmodpg^b \bmod p

      Shared secret: (ga)b=(gb)a=gabmodp(g^a)^b = (g^b)^a = g^{ab} \bmod p.

      Security relies on the difficulty of computing discrete logarithms.

      Fermat’s Theorem and Wilson’s Theorem

      Wilson’s Theorem

      (p1)!1(modp)(p-1)! \equiv -1 \pmod{p} iff pp is prime.

      Connection

      In our proof of Fermat’s Little Theorem, we used that (p1)!(p-1)! is coprime to pp. Wilson’s theorem gives its exact value.

      Generalisations

      Euler’s Theorem

      For aa coprime to nn: aϕ(n)1(modn)a^{\phi(n)} \equiv 1 \pmod{n}, where ϕ(n)\phi(n) is Euler’s totient function.

      When nn is prime, ϕ(n)=n1\phi(n) = n-1, giving Fermat’s Little Theorem.


      Summary

      • Fermat’s Little Theorem: ap11(modp)a^{p-1} \equiv 1 \pmod{p} for prime pp and pap \nmid a
      • Proof uses permutation argument and cancellation
      • Applications: fast modular exponentiation, cryptography (Diffie-Hellman), primality testing
      • Generalised by Euler’s theorem to composite moduli
    • Principles of Induction

      Simple Induction

      Principle

      To prove nN.P(n)\forall n \in \mathbb{N}. P(n):

      1. Base case: Prove P(0)P(0)
      2. Inductive step: Prove that for arbitrary nNn \in \mathbb{N}, P(n)P(n+1)P(n) \Rightarrow P(n+1)

      If both hold, then by the principle of mathematical induction, P(n)P(n) is true for all nNn \in \mathbb{N}.

      Why It Works

      Induction is justified by the well-ordering of N\mathbb{N}.

      Suppose P(0)P(0) holds and P(n)P(n+1)P(n) \Rightarrow P(n+1) for all nn, but some kk fails P(k)P(k).

      Let S={nN:¬P(n)}S = \{n \in \mathbb{N} : \neg P(n)\}. By assumption, SS \neq \emptyset.

      By well-ordering, SS has a least element mm.

      Since P(0)P(0) holds, m>0m > 0.

      Thus m1Nm - 1 \in \mathbb{N} and m1Sm - 1 \notin S, so P(m1)P(m-1) holds.

      By the inductive step, P(m)P(m) holds. Contradiction.

      Example: Sum Formula

      Statement: i=0n1i=n(n1)2\displaystyle\sum_{i=0}^{n-1} i = \frac{n(n-1)}{2} for all nNn \in \mathbb{N}.

      Base case (n=0n = 0): Empty sum = 0. And 0(1)2=0\frac{0 \cdot (-1)}{2} = 0.

      Inductive step: Assume i=0n1i=n(n1)2\displaystyle\sum_{i=0}^{n-1} i = \frac{n(n-1)}{2}.

      Then: i=0ni=n(n1)2+n=n(n1)+2n2=n(n+1)2\sum_{i=0}^{n} i = \frac{n(n-1)}{2} + n = \frac{n(n-1) + 2n}{2} = \frac{n(n+1)}{2}

      This is (n+1)(n)2\frac{(n+1)(n)}{2}, establishing P(n+1)P(n+1).

      Induction from a Basis

      Principle

      To prove nk.P(n)\forall n \geq k. P(n) for some fixed kk:

      1. Base case: Prove P(k)P(k)
      2. Inductive step: Prove P(n)P(n+1)P(n) \Rightarrow P(n+1) for arbitrary nkn \geq k

      When to Use

      When the property P(n)P(n) is only meaningful or true for nkn \geq k.

      Example

      Statement: 2n>n22^n > n^2 for all n5n \geq 5.

      Base case (n=5n = 5): 25=32>25=522^5 = 32 > 25 = 5^2.

      Inductive step: Assume 2n>n22^n > n^2 for n5n \geq 5.

      Then 2n+1=22n>2n22^{n+1} = 2 \cdot 2^n > 2n^2.

      For n5n \geq 5: 2n2(n+1)2=2n2n22n1=n22n125101=14>02n^2 - (n+1)^2 = 2n^2 - n^2 - 2n - 1 = n^2 - 2n - 1 \geq 25 - 10 - 1 = 14 > 0.

      So 2n2>(n+1)22n^2 > (n+1)^2, giving 2n+1>(n+1)22^{n+1} > (n+1)^2.

      Strong Induction

      Principle

      To prove nN.P(n)\forall n \in \mathbb{N}. P(n):

      1. Inductive step: Prove that for arbitrary nn: (m<n.P(m))P(n)\left(\forall m < n. P(m)\right) \Rightarrow P(n)

      No explicit base case needed: when n=0n = 0, the hypothesis "m<0.P(m)\forall m < 0. P(m)" is vacuously true.

      When to Use

      When proving P(n)P(n) requires multiple previous cases, not just P(n1)P(n-1).

      Example: Fibonacci

      The Fibonacci sequence: F0=0F_0 = 0, F1=1F_1 = 1, Fn+1=Fn+Fn1F_{n+1} = F_n + F_{n-1}.

      Statement: Fn<2nF_n < 2^n for all nNn \in \mathbb{N}.

      Strong induction:

      Assume m<n.Fm<2m\forall m < n. F_m < 2^m.

      If n1n \leq 1: F0=0<1=20F_0 = 0 < 1 = 2^0 and F1=1<2=21F_1 = 1 < 2 = 2^1.

      For n2n \geq 2: Fn=Fn1+Fn2<2n1+2n2=342n<2nF_n = F_{n-1} + F_{n-2} < 2^{n-1} + 2^{n-2} = \frac{3}{4} \cdot 2^n < 2^n

      Common Pitfalls

      Wrong Inductive Hypothesis

      Trap: Assuming P(n1)P(n-1) when P(n2)P(n-2) is also needed.

      This happens in Fibonacci-style recurrences. Use strong induction instead.

      Forgetting the Base Case

      Trap: Proving P(n)P(n+1)P(n) \Rightarrow P(n+1) without checking PP holds somewhere.

      Assuming What You Prove

      Trap: The inductive step must assume P(n)P(n) and derive P(n+1)P(n+1).

      Do not assume P(n+1)P(n+1).

      Inductive Step Fails for Small nn

      Trap: Check the inductive step works for the transition from base case.

      Example of Failure

      False claim: All horses are the same colour.

      “Proof” by induction on herd size nn:

      • Base (n=1n = 1): Trivial.
      • Inductive step: If all horses in a herd of nn have the same colour, then in a herd of n+1n+1:
        • First nn horses have same colour (by IH)
        • Last nn horses have same colour (by IH)
        • Therefore all n+1n+1 horses have same colour.

      Error: When n=1n = 1, there is no overlap between “first nn” and “last nn” (both are single horses). The argument fails.


      Summary

      • Simple induction: prove P(0)P(0), then P(n)P(n+1)P(n) \Rightarrow P(n+1)
      • Induction from basis: P(k)P(k), then P(n)P(n+1)P(n) \Rightarrow P(n+1) for nkn \geq k
      • Strong induction: (m<n.P(m))P(n)(\forall m < n. P(m)) \Rightarrow P(n)
      • Strong induction needed for recurrences depending on multiple predecessors
    • Abstract Algebra: Monoids, Groups, Semirings, and Fields

      The Hierarchy of Algebraic Structures

      Abstract algebra studies mathematical structures by specifying the operations and properties they must satisfy. Each structure adds more requirements, making it more powerful but more restrictive.

      Hierarchy of algebraic structures


      Monoids

      Definition

      A monoid is a triple (S,,e)(S, \bullet, e) where:

      • SS is a set
      • :S×SS\bullet : S \times S \to S is a binary operation (closed)
      • eSe \in S is the identity element

      satisfying:

      1. Associativity: (xy)z=x(yz)(x \bullet y) \bullet z = x \bullet (y \bullet z) for all x,y,zSx, y, z \in S
      2. Identity: ex=x=xee \bullet x = x = x \bullet e for all xSx \in S

      Examples

      MonoidOperationIdentity
      (N,+,0)(\mathbb{N}, +, 0)Addition00
      (N,×,1)(\mathbb{N}, \times, 1)Multiplication11
      (Σ,,ε)(\Sigma^*, \cdot, \varepsilon)String concatenationε\varepsilon
      (Rel(A,A),,idA)(\text{Rel}(A, A), \circ, \text{id}_A)Relational compositionidA\text{id}_A
      (P(A),,)(\mathcal{P}(A), \cup, \emptyset)Union\emptyset
      (P(A),,A)(\mathcal{P}(A), \cap, A)IntersectionAA

      Commutative Monoid

      A monoid is commutative (or abelian) if xy=yxx \bullet y = y \bullet x for all x,ySx, y \in S.

      Examples: (N,+,0)(\mathbb{N}, +, 0), (P(A),,)(\mathcal{P}(A), \cup, \emptyset).

      Non-example: (Σ,,ε)(\Sigma^*, \cdot, \varepsilon) is NOT commutative (abbaab \neq ba in general).

      Examinable: Endo-Relations Form a Monoid

      Theorem: For any set AA, the endo-relations (Rel(A,A),,idA)(\text{Rel}(A, A), \circ, \text{id}_A) form a monoid.

      Proof:

      • Closure: Composition of relations R:AAR : A \nrightarrow A and S:AAS : A \nrightarrow A yields SR:AAS \circ R : A \nrightarrow A.
      • Associativity: Already proved: (TS)R=T(SR)(T \circ S) \circ R = T \circ (S \circ R).
      • Identity: idAR=R=RidA\text{id}_A \circ R = R = R \circ \text{id}_A.

      Groups

      Definition

      A group is a monoid (G,,e)(G, \bullet, e) where every element has an inverse.

      For each xGx \in G, there exists x1Gx^{-1} \in G such that:

      xx1=e=x1xx \bullet x^{-1} = e = x^{-1} \bullet x

      Examples

      GroupOperationIdentityInverse
      (Z,+,0)(\mathbb{Z}, +, 0)Addition00x-x
      (Q{0},×,1)(\mathbb{Q} \setminus \{0\}, \times, 1)Multiplication111/x1/x
      (Zn,+,0)(\mathbb{Z}_n, +, 0) mod nnAddition mod nn00nxn - x
      Symmetric group SnS_nPermutation compositionIdentity permutationInverse permutation

      Abelian Group

      A group is abelian if the operation is commutative.

      Key Property

      In a group, equations are always solvable: for any a,bGa, b \in G, there exists a unique xx with ax=ba \bullet x = b (namely x=a1bx = a^{-1} \bullet b).

      Non-examples

      • (N,+,0)(\mathbb{N}, +, 0) is NOT a group (no additive inverses)
      • (Z,×,1)(\mathbb{Z}, \times, 1) is NOT a group (most elements lack multiplicative inverses)

      Semirings

      Definition

      A semiring is a structure (S,0,,1,)(S, 0, \oplus, 1, \otimes) where:

      1. (S,0,)(S, 0, \oplus) is a commutative monoid (addition)
      2. (S,1,)(S, 1, \otimes) is a monoid (multiplication)
      3. Distributivity: x(yz)=(xy)(xz)x \otimes (y \oplus z) = (x \otimes y) \oplus (x \otimes z) and (xy)z=(xz)(yz)(x \oplus y) \otimes z = (x \otimes z) \oplus (y \otimes z)
      4. Annihilation: 0x=0=x00 \otimes x = 0 = x \otimes 0

      Examples

      SemiringAddition \oplusMultiply \otimesZeroOne
      (N,+,×)(\mathbb{N}, +, \times)++×\times0011
      Boolean semiring B\mathbb{B}\lor (OR)\land (AND)0011
      Min-plus (tropical)min\min++\infty00
      Max-plusmax\max++-\infty00

      The Boolean Semiring

      B=({0,1},0,,1,)\mathbb{B} = (\{0, 1\}, 0, \lor, 1, \land)

      This is the foundation for:

      • Relational composition via matrix multiplication
      • Reachability in graphs

      The Min-Plus Semiring (Tropical)

      (R{},,min,0,+)(\mathbb{R} \cup \{\infty\}, \infty, \min, 0, +)

      Applications:

      • Shortest path algorithms
      • Dynamic programming optimisation

      Boolean vs Min-Plus semiring comparison


      Rings

      Definition

      A ring is a semiring where (S,0,)(S, 0, \oplus) is an abelian group.

      Every element has an additive inverse.

      Examples

      • (Z,+,×)(\mathbb{Z}, +, \times): integers (additive inverse is negation)
      • (Zn,+,×)(\mathbb{Z}_n, +, \times): integers mod nn
      • Polynomials over Z\mathbb{Z}

      Non-example

      (N,+,×)(\mathbb{N}, +, \times) is NOT a ring (no additive inverses).


      Fields

      Definition

      A field is a commutative ring where every non-zero element has a multiplicative inverse.

      (F,0,+,1,×)(F, 0, +, 1, \times) satisfies:

      1. (F,0,+)(F, 0, +) is an abelian group
      2. (F{0},1,×)(F \setminus \{0\}, 1, \times) is an abelian group
      3. Distributivity holds

      Examples

      • Q\mathbb{Q}: rational numbers
      • R\mathbb{R}: real numbers
      • C\mathbb{C}: complex numbers
      • Zp\mathbb{Z}_p: integers mod pp, where pp is prime

      Key Theorem

      Theorem: Zp\mathbb{Z}_p (integers mod pp) is a field if and only if pp is prime.

      Proof:

      • If pp is prime and a≢0(modp)a \not\equiv 0 \pmod{p}, then gcd(a,p)=1\gcd(a, p) = 1.
      • By Bézout’s identity, x,y\exists x, y with ax+py=1ax + py = 1.
      • Thus ax1(modp)ax \equiv 1 \pmod{p}, so aa has a multiplicative inverse.
      • If pp is composite, say p=abp = ab with 1<a,b<p1 < a, b < p, then a,ba, b are zero divisors and cannot have inverses.

      Why This Matters for Cryptography

      Zp\mathbb{Z}_p being a field is essential for:

      • Diffie-Hellman key exchange: relies on exponentiation in Zp\mathbb{Z}_p^*
      • RSA: requires computing inverses mod φ(n)\varphi(n)

      Summary Table

      StructureAdditionMultiplicationKey Property
      MonoidAssociative + IdentityMinimal structure
      GroupInverses existEquations solvable
      SemiringCommutative monoidMonoidMatrix multiplication works
      RingAbelian groupMonoidCan subtract
      FieldAbelian groupAbelian group (except 0)Can divide

      Tripos Questions

      2016 P2 Q6: Prove that the set of all strings over alphabet {a,b}\{a, b\} under concatenation forms a monoid. Is it a group?

      2020 P2 Q9: Show that Zp\mathbb{Z}_p is a field when pp is prime. Explain why ap11(modp)a^{p-1} \equiv 1 \pmod{p} for a≢0(modp)a \not\equiv 0 \pmod{p}.


      Summary

      • Monoid: set + operation + identity + associativity
      • Group: monoid + inverses
      • Semiring: two operations, distributivity, additive identity annihilates
      • Ring: semiring + additive inverses
      • Field: ring + multiplicative inverses (except zero)
      • Zp\mathbb{Z}_p is a field iff pp is prime (foundational for cryptography)
  • Sets, Relations, and Functions

    Set theory, Venn diagrams, relations, path closures, and function classifications

    • Sets: Basics, Extensionality, and Russell's Paradox

      What is a Set?

      A set is an unordered collection of distinct objects. The objects in a set are called its elements or members.

      Membership

      We write xAx \in A to mean ”xx is an element of AA” and xAx \notin A to mean ”xx is not an element of AA”.

      Examples

      • {1,2,3}\{1, 2, 3\}: the set containing 1, 2, and 3
      • N={0,1,2,3,}\mathbb{N} = \{0, 1, 2, 3, \ldots\}: the set of natural numbers
      • \emptyset or {}\{\}: the empty set
      • {xZ:x2<10}={3,2,1,0,1,2,3}\{x \in \mathbb{Z} : x^2 < 10\} = \{-3, -2, -1, 0, 1, 2, 3\}

      Extensionality

      The Axiom of Extensionality

      Two sets are equal if and only if they contain exactly the same elements:

      A=B    x.(xAxB)A = B \iff \forall x. (x \in A \Leftrightarrow x \in B)

      Consequences

      • Order does not matter: {1,2,3}={3,2,1}\{1, 2, 3\} = \{3, 2, 1\}
      • Repetition does not matter: {1,1,2,2}={1,2}\{1, 1, 2, 2\} = \{1, 2\}
      • A set is determined entirely by its membership

      Proof Strategy

      To prove A=BA = B: show that every element of AA is in BB, and every element of BB is in AA.

      Subsets and Supersets

      Definitions

      • ABA \subseteq B (AA is a subset of BB): every element of AA is in BB AB    xA.xBA \subseteq B \iff \forall x \in A. x \in B

      • ABA \subsetneq B (AA is a proper subset of BB): ABA \subseteq B and ABA \neq B

      • ABA \supseteq B (AA is a superset of BB): BAB \subseteq A

      Properties

      • Reflexivity: AAA \subseteq A
      • Antisymmetry: ABBAA=BA \subseteq B \land B \subseteq A \Rightarrow A = B
      • Transitivity: ABBCACA \subseteq B \land B \subseteq C \Rightarrow A \subseteq C

      The Empty Set

      The empty set \emptyset is the unique set with no elements.

      x.x\forall x. x \notin \emptyset

      Properties

      • A\emptyset \subseteq A for any set AA (vacuously true)
      • =0|\emptyset| = 0

      Separation (Restricted Comprehension)

      Naive Comprehension (Problematic)

      The naive comprehension principle would allow: for any property PP, form {x:P(x)}\{x : P(x)\}.

      Russell’s Paradox

      Consider the “set” R={x:xx}R = \{x : x \notin x\}.

      Is RRR \in R?

      • If RRR \in R: then RR satisfies the defining property, so RRR \notin R. Contradiction.
      • If RRR \notin R: then RR satisfies the defining property, so RRR \in R. Contradiction.

      This is a genuine paradox, showing naive comprehension is inconsistent.

      Separation Schema

      Given a set AA and a property PP, we can form:

      {xA:P(x)}\{x \in A : P(x)\}

      This is restricted comprehension or separation: we can only form subsets of existing sets.

      Set Operations

      Union

      AB={x:xAxB}A \cup B = \{x : x \in A \lor x \in B\}

      Intersection

      AB={x:xAxB}A \cap B = \{x : x \in A \land x \in B\}

      Set Difference

      AB={xA:xB}A \setminus B = \{x \in A : x \notin B\}

      Complement (Relative to a Universe UU)

      Ac=UA={xU:xA}A^c = U \setminus A = \{x \in U : x \notin A\}

      Laws of Set Algebra

      Identity Laws

      A=AA \cup \emptyset = A AU=AA \cap U = A

      Domination Laws

      AU=UA \cup U = U A=A \cap \emptyset = \emptyset

      Idempotent Laws

      AA=AA \cup A = A AA=AA \cap A = A

      De Morgan’s Laws

      (AB)c=AcBc(A \cup B)^c = A^c \cap B^c (AB)c=AcBc(A \cap B)^c = A^c \cup B^c

      Distributive Laws

      A(BC)=(AB)(AC)A \cap (B \cup C) = (A \cap B) \cup (A \cap C) A(BC)=(AB)(AC)A \cup (B \cap C) = (A \cup B) \cap (A \cup C)


      Summary

      • Sets are collections determined entirely by membership (extensionality)
      • ABA \subseteq B means every element of AA is in BB
      • Russell’s paradox shows naive comprehension is inconsistent
      • Use separation: {xA:P(x)}\{x \in A : P(x)\} for safe set construction
      • Sets form a Boolean algebra under \cup, \cap, complement
    • Powersets and Boolean Algebra

      Powersets

      Definition

      The powerset of a set AA, written P(A)\mathcal{P}(A) or 2A2^A, is the set of all subsets of AA.

      P(A)={S:SA}\mathcal{P}(A) = \{S : S \subseteq A\}

      Examples

      • P()={}\mathcal{P}(\emptyset) = \{\emptyset\}
      • P({1})={,{1}}\mathcal{P}(\{1\}) = \{\emptyset, \{1\}\}
      • P({1,2})={,{1},{2},{1,2}}\mathcal{P}(\{1, 2\}) = \{\emptyset, \{1\}, \{2\}, \{1, 2\}\}

      Cardinality

      For finite sets:

      P(A)=2A|\mathcal{P}(A)| = 2^{|A|}

      Proof: Each element of AA either is or is not in a subset. For A=n|A| = n, there are nn independent binary choices, giving 2n2^n subsets.

      Visualising Powersets

      Hasse Diagrams

      A Hasse diagram shows P(A)\mathcal{P}(A) ordered by \subseteq:

      • Elements are arranged with \emptyset at the bottom
      • Larger sets are drawn above subsets
      • Edges connect sets that differ by exactly one element

      Hasse diagram for P({1,2,3})

      Venn Diagrams

      Venn diagrams visualise set relationships in the plane.

      Universal set UU is shown as a rectangle. Sets AA, BB are shown as overlapping circles within UU.

      Venn diagram for A union B

      The Powerset Boolean Algebra

      Structure

      (P(A),,,c,,A)(\mathcal{P}(A), \cup, \cap, ^c, \emptyset, A) forms a Boolean algebra:

      OperationSymbolCorresponds to
      Join\cupOR
      Meet\capAND
      Complementc^cNOT
      Bottom\emptysetFalse
      TopAATrue

      Boolean Algebra Laws

      Commutativity: AB=BAA \cup B = B \cup A AB=BAA \cap B = B \cap A

      Associativity: (AB)C=A(BC)(A \cup B) \cup C = A \cup (B \cup C) (AB)C=A(BC)(A \cap B) \cap C = A \cap (B \cap C)

      Absorption: A(AB)=AA \cup (A \cap B) = A A(AB)=AA \cap (A \cup B) = A

      Identity: A=AA \cup \emptyset = A AU=AA \cap U = A

      Complement: AAc=UA \cup A^c = U AAc=A \cap A^c = \emptyset

      The Analog with Propositional Logic

      Set TheoryLogic
      \cup\lor
      \cap\land
      c^c¬\neg
      \subseteq\Rightarrow
      \emptysetFalse
      UUTrue

      This correspondence is not coincidental. The Boolean algebra of sets is a model of propositional logic, and vice versa.

      Characteristic (Indicator) Functions

      Definition

      For SAS \subseteq A, the characteristic function (or indicator function) χS:A{0,1}\chi_S : A \to \{0, 1\} is:

      χS(x)={1if xS0if xS\chi_S(x) = \begin{cases} 1 & \text{if } x \in S \\ 0 & \text{if } x \notin S \end{cases}

      Correspondence

      There is a bijection between P(A)\mathcal{P}(A) and the set of functions A{0,1}A \to \{0, 1\}:

      P(A){f:A{0,1}}\mathcal{P}(A) \cong \{f : A \to \{0, 1\}\}

      This gives an alternative proof that P(A)=2A|\mathcal{P}(A)| = 2^{|A|} for finite AA.


      Summary

      • P(A)\mathcal{P}(A) is the set of all subsets of AA
      • P(A)=2A|\mathcal{P}(A)| = 2^{|A|} for finite AA
      • P(A)\mathcal{P}(A) forms a Boolean algebra with \cup, \cap, c^c
      • Hasse diagrams show inclusion structure; Venn diagrams show overlap relations
      • Characteristic functions establish P(A)(A{0,1})\mathcal{P}(A) \cong (A \to \{0, 1\})
    • Pairing and Cartesian Products

      Ordered Pairs

      Definition

      An ordered pair (a,b)(a, b) is a mathematical structure that:

      1. Contains exactly two elements
      2. Has a specified order: first component aa, second component bb

      Key Property

      (a,b)=(c,d)    a=cb=d(a, b) = (c, d) \iff a = c \land b = d

      This distinguishes ordered pairs from sets: {a,b}={b,a}\{a, b\} = \{b, a\} but (a,b)(b,a)(a, b) \neq (b, a) (unless a=ba = b).

      Kuratowski Definition (Encoding)

      We can define ordered pairs purely in terms of sets:

      (a,b)={{a},{a,b}}(a, b) = \{\{a\}, \{a, b\}\}

      Verification: If (a,b)=(c,d)(a, b) = (c, d), then {{a},{a,b}}={{c},{c,d}}\{\{a\}, \{a, b\}\} = \{\{c\}, \{c, d\}\}.

      • If a=ba = b, then {{a},{a,b}}={{a}}\{\{a\}, \{a, b\}\} = \{\{a\}\}, so {c}={a}\{c\} = \{a\}, hence c=a=bc = a = b.
      • If aba \neq b, then {a}{{c},{c,d}}\{a\} \in \{\{c\}, \{c, d\}\} requires {a}={c}\{a\} = \{c\}, so a=ca = c.
        • Then {a,b}={a,d}\{a, b\} = \{a, d\}, and since aba \neq b, we have b=db = d.

      Cartesian Product

      Definition

      The Cartesian product of sets AA and BB is:

      A×B={(a,b):aAbB}A \times B = \{(a, b) : a \in A \land b \in B\}

      Examples

      • {1,2}×{a,b}={(1,a),(1,b),(2,a),(2,b)}\{1, 2\} \times \{a, b\} = \{(1, a), (1, b), (2, a), (2, b)\}
      • R×R=R2\mathbb{R} \times \mathbb{R} = \mathbb{R}^2 (the Euclidean plane)
      • {x}×A={(x,a):aA}\{x\} \times A = \{(x, a) : a \in A\}

      Cardinality

      For finite sets:

      A×B=AB|A \times B| = |A| \cdot |B|

      Properties of Cartesian Products

      Non-commutativity

      A×BB×A in generalA \times B \neq B \times A \text{ in general}

      Unless A=BA = B or one of them is empty.

      Distributivity over Union

      A×(BC)=(A×B)(A×C)A \times (B \cup C) = (A \times B) \cup (A \times C) (AB)×C=(A×C)(B×C)(A \cup B) \times C = (A \times C) \cup (B \times C)

      Distributivity over Intersection

      A×(BC)=(A×B)(A×C)A \times (B \cap C) = (A \times B) \cap (A \times C) (AB)×C=(A×C)(B×C)(A \cap B) \times C = (A \times C) \cap (B \times C)

      Interaction with Empty Set

      A×==×AA \times \emptyset = \emptyset = \emptyset \times A

      n-tuples

      Definition

      An n-tuple (a1,a2,,an)(a_1, a_2, \ldots, a_n) generalises ordered pairs.

      We can encode n-tuples using nested pairs:

      (a1,a2,,an)=(a1,(a2,(,(an1,an))))(a_1, a_2, \ldots, a_n) = (a_1, (a_2, (\ldots, (a_{n-1}, a_n)\ldots)))

      n-fold Cartesian Product

      An=A×A××An times={(a1,,an):aiA}A^n = \underbrace{A \times A \times \cdots \times A}_{n \text{ times}} = \{(a_1, \ldots, a_n) : a_i \in A\}

      Examples

      • R2\mathbb{R}^2: the plane (2D coordinates)
      • R3\mathbb{R}^3: 3D space
      • Zn\mathbb{Z}^n: integer lattice in nn dimensions

      Disjoint Union (Coproduct)

      Motivation

      The union ABA \cup B is not ideal for “combining” sets because:

      • If AA and BB overlap, we lose information about origin
      • We cannot distinguish aABa \in A \cap B from aAa \in A vs aBa \in B

      Definition

      The disjoint union (or coproduct, or sum) of AA and BB is:

      A+B=({0}×A)({1}×B)A + B = (\{0\} \times A) \cup (\{1\} \times B)

      Each element is “tagged” with its origin.

      Examples

      • {a,b}+{b,c}={(0,a),(0,b)}{(1,b),(1,c)}\{a, b\} + \{b, c\} = \{(0, a), (0, b)\} \cup \{(1, b), (1, c)\}
      • Even though bb appears in both, (0,b)(1,b)(0, b) \neq (1, b)

      Cardinality

      A+B=A+B|A + B| = |A| + |B|

      This holds without restriction (unlike union, where AB=A+BAB|A \cup B| = |A| + |B| - |A \cap B|).


      Summary

      • Ordered pairs (a,b)(a, b) satisfy (a,b)=(c,d)    a=cb=d(a, b) = (c, d) \iff a = c \land b = d
      • Cartesian product A×B={(a,b):aA,bB}A \times B = \{(a, b) : a \in A, b \in B\}
      • A×B=AB|A \times B| = |A| \cdot |B| and A+B=A+B|A + B| = |A| + |B|
      • Disjoint union A+BA + B tags elements to preserve origin information
    • Unions and Intersections

      Binary Operations

      Union

      AB={x:xAxB}A \cup B = \{x : x \in A \lor x \in B\}

      Intersection

      AB={x:xAxB}A \cap B = \{x : x \in A \land x \in B\}

      Properties

      PropertyUnionIntersection
      CommutativeAB=BAA \cup B = B \cup AAB=BAA \cap B = B \cap A
      Associative(AB)C=A(BC)(A \cup B) \cup C = A \cup (B \cup C)(AB)C=A(BC)(A \cap B) \cap C = A \cap (B \cap C)
      IdentityA=AA \cup \emptyset = AAU=AA \cap U = A
      IdempotentAA=AA \cup A = AAA=AA \cap A = A

      Distributive Laws

      A(BC)=(AB)(AC)A \cap (B \cup C) = (A \cap B) \cup (A \cap C) A(BC)=(AB)(AC)A \cup (B \cap C) = (A \cup B) \cap (A \cup C)

      Proof of the First

      xA(BC)    xA(xBxC)x \in A \cap (B \cup C) \iff x \in A \land (x \in B \lor x \in C)     (xAxB)(xAxC)\iff (x \in A \land x \in B) \lor (x \in A \land x \in C)     x(AB)(AC)\iff x \in (A \cap B) \cup (A \cap C)

      Big Unions and Intersections

      Notation

      For an indexed family of sets {Ai}iI\{A_i\}_{i \in I}:

      iIAi={x:iI.xAi}\bigcup_{i \in I} A_i = \{x : \exists i \in I. x \in A_i\}

      iIAi={x:iI.xAi}\bigcap_{i \in I} A_i = \{x : \forall i \in I. x \in A_i\}

      Examples

      • nN{0,1,,n}=N\displaystyle\bigcup_{n \in \mathbb{N}} \{0, 1, \ldots, n\} = \mathbb{N}
      • nN{n,n+1,n+2,}=\displaystyle\bigcap_{n \in \mathbb{N}} \{n, n+1, n+2, \ldots\} = \emptyset
      • nN{n}=N\displaystyle\bigcup_{n \in \mathbb{N}} \{n\} = \mathbb{N} (each {n}\{n\} is a singleton)

      De Morgan’s Laws for Big Operations

      (iIAi)c=iIAic\left(\bigcup_{i \in I} A_i\right)^c = \bigcap_{i \in I} A_i^c (iIAi)c=iIAic\left(\bigcap_{i \in I} A_i\right)^c = \bigcup_{i \in I} A_i^c

      Verification

      x(iIAi)c    ¬(iI.xAi)x \in (\bigcup_{i \in I} A_i)^c \iff \neg(\exists i \in I. x \in A_i)     iI.xAi\iff \forall i \in I. x \notin A_i     iI.xAic\iff \forall i \in I. x \in A_i^c     xiIAic\iff x \in \bigcap_{i \in I} A_i^c

      Distributivity of Big Operations

      Big Union over Intersection

      A(iIBi)=iI(ABi)A \cap \left(\bigcup_{i \in I} B_i\right) = \bigcup_{i \in I} (A \cap B_i)

      Big Intersection over Union

      A(iIBi)=iI(ABi)A \cup \left(\bigcap_{i \in I} B_i\right) = \bigcap_{i \in I} (A \cup B_i)

      Empty Index Set

      Convention

      When I=I = \emptyset:

      iAi=\bigcup_{i \in \emptyset} A_i = \emptyset iAi=U\bigcap_{i \in \emptyset} A_i = U

      Rationale: i\exists i \in \emptyset is always false, so the union is empty.

      For intersection: i\forall i \in \emptyset is vacuously true, so every xx satisfies the condition. The intersection is the entire universe UU.

      Set Difference

      Definition

      AB={xA:xB}A \setminus B = \{x \in A : x \notin B\}

      Properties

      • AB=ABcA \setminus B = A \cap B^c
      • AA=A \setminus A = \emptyset
      • A=AA \setminus \emptyset = A
      • A=\emptyset \setminus A = \emptyset

      Symmetric Difference

      AB=(AB)(BA)A \triangle B = (A \setminus B) \cup (B \setminus A)

      This is the set of elements in exactly one of AA or BB.


      Summary

      • \cup and \cap are commutative, associative, and idempotent
      • \cup distributes over \cap and vice versa
      • Big unions iIAi\bigcup_{i \in I} A_i generalise to indexed families
      • De Morgan’s laws apply to complemented big unions/intersections
    • Disjoint Unions

      Motivation: The Problem with Union

      Given sets AA and BB, the ordinary union ABA \cup B loses information:

      • If xABx \in A \cap B, we cannot tell if xx came from AA, from BB, or from both
      • The size is AB=A+BAB|A \cup B| = |A| + |B| - |A \cap B|, not simply A+B|A| + |B|

      Definition of Disjoint Union

      The disjoint union (also called coproduct or sum) of AA and BB is:

      A+B=({0}×A)({1}×B)A + B = (\{0\} \times A) \cup (\{1\} \times B)

      Each element is “tagged” with its origin: 00 for elements from AA, 11 for elements from BB.

      Alternative Notations

      • A⨿BA \amalg B (amalgamation symbol)
      • ABA \sqcup B
      • ABA \oplus B

      Examples

      Example 1

      A={a,b}A = \{a, b\}, B={b,c}B = \{b, c\}.

      • AB={a,b,c}A \cup B = \{a, b, c\} (ordinary union, bb appears once)
      • A+B={(0,a),(0,b),(1,b),(1,c)}A + B = \{(0, a), (0, b), (1, b), (1, c)\} (disjoint union, bb appears twice with different tags)

      Example 2

      A={1,2,3}A = \{1, 2, 3\}, B={x,y}B = \{x, y\}.

      A+B={(0,1),(0,2),(0,3),(1,x),(1,y)}A + B = \{(0, 1), (0, 2), (0, 3), (1, x), (1, y)\}

      Properties

      Cardinality

      A+B=A+B|A + B| = |A| + |B|

      This is the key advantage: the size adds without any correction.

      Injective Maps

      There are natural injection maps:

      • ιA:AA+B\iota_A : A \to A + B given by ιA(a)=(0,a)\iota_A(a) = (0, a)
      • ιB:BA+B\iota_B : B \to A + B given by ιB(b)=(1,b)\iota_B(b) = (1, b)

      Universal Property

      Given any set CC and functions f:ACf : A \to C, g:BCg : B \to C, there is a unique function [f,g]:A+BC[f, g] : A + B \to C such that:

      [f,g]ιA=fand[f,g]ιB=g[f, g] \circ \iota_A = f \quad \text{and} \quad [f, g] \circ \iota_B = g

      Explicitly: [f,g](0,a)=f(a)[f, g](0, a) = f(a) and [f,g](1,b)=g(b)[f, g](1, b) = g(b).

      Universal property of disjoint union

      Case Analysis

      The disjoint union enables case analysis in proofs:

      To define a function h:A+BCh : A + B \to C, define:

      1. A function hA:ACh_A : A \to C for the “left” case
      2. A function hB:BCh_B : B \to C for the “right” case

      Then h=[hA,hB]h = [h_A, h_B].

      n-ary Disjoint Union

      For sets A1,,AnA_1, \ldots, A_n:

      A1+A2++An={(i,a):i{1,,n},aAi}A_1 + A_2 + \cdots + A_n = \{(i, a) : i \in \{1, \ldots, n\}, a \in A_i\}

      Cardinality

      i=1nAi=i=1nAi\left|\sum_{i=1}^{n} A_i\right| = \sum_{i=1}^{n} |A_i|

      Product vs Sum

      Product ×\timesSum ++
      Pair (a,b)(a, b) with bothTagged (i,a)(i, a) with one
      $A \times B
      Projection mapsInjection maps
      Universal: maps outUniversal: maps in

      These are dual constructions in category theory.


      Summary

      • Disjoint union A+BA + B tags elements to preserve origin
      • A+B=A+B|A + B| = |A| + |B| without intersection correction
      • Natural injections ιA\iota_A, ιB\iota_B embed AA, BB into the sum
      • Universal property: maps A+BCA + B \to C correspond to pairs of maps ACA \to C, BCB \to C
    • Relations: Introduction

      What is a Relation?

      Definition

      A binary relation RR from a set AA to a set BB is a subset RA×BR \subseteq A \times B.

      We write:

      • R:ABR : A \nrightarrow B to indicate a relation from AA to BB
      • aRbaRb or (a,b)R(a, b) \in R to indicate RR relates aa and bb

      Intuition

      A relation describes how elements of AA are connected to elements of BB. Unlike a function, a single aAa \in A can be related to multiple bBb \in B, or to none at all.

      Internal diagram of a relation

      Relational Extensionality

      Two relations are equal iff they relate the same pairs:

      R=S    aA.bB.(aRbaSb)R = S \iff \forall a \in A. \forall b \in B. (aRb \Leftrightarrow aSb)

      Examples of Relations

      The Empty Relation

      For any sets A,BA, B, the empty relation A×B\emptyset \subseteq A \times B relates nothing.

      ab is never truea \emptyset\, b \text{ is never true}

      The Full Relation

      The full relation A×BA×BA \times B \subseteq A \times B relates everything.

      a(A×B)b for all aA,bBa (A \times B) b \text{ for all } a \in A, b \in B

      The Identity Relation

      For a set AA, the identity relation idAA×A\text{id}_A \subseteq A \times A:

      aidAa    a=aa \,\text{id}_A\, a' \iff a = a'

      Examples from Computer Science

      Program specification: The relation sq:R0R\text{sq} : \mathbb{R}_{\geq 0} \nrightarrow \mathbb{R} given by xsqy    x=y2x \,\text{sq}\, y \iff x = y^2.

      Operational semantics: A transition relation S:EES : E \nrightarrow E on machine states where eSeeSe' means “state ee can transition to ee'”.

      Network connectivity: A relation C:NNC : N \nrightarrow N on network nodes, where aCbaCb means “node aa is directly connected to node bb”.

      Database relations: A table relating Movies, Directors, Years, and People.

      Special Relations on a Single Set

      A relation RA×AR \subseteq A \times A is called an endo-relation or relation on AA.

      Reflexivity

      RR is reflexive if aA.aRa\forall a \in A. aRa.

      Equivalently: idAR\text{id}_A \subseteq R.

      Symmetry

      RR is symmetric if a,bA.(aRbbRa)\forall a, b \in A. (aRb \Rightarrow bRa).

      Transitivity

      RR is transitive if a,b,cA.(aRbbRcaRc)\forall a, b, c \in A. (aRb \land bRc \Rightarrow aRc).

      Antisymmetry

      RR is antisymmetric if a,bA.(aRbbRaa=b)\forall a, b \in A. (aRb \land bRa \Rightarrow a = b).

      Examples

      RelationReflexiveSymmetricTransitiveAntisymmetric
      a=ba = b on any setYesYesYesYes
      aba \leq b on R\mathbb{R}YesNoYesYes
      a<ba < b on R\mathbb{R}NoNoYesYes
      aba \mid b on Z\mathbb{Z}YesNoYesNo
      ”is a sibling of” on peopleNoYesNoNo

      Visualising Relations

      Internal Diagrams

      Draw elements of AA on the left, elements of BB on the right. Draw arrows from aa to bb whenever aRbaRb.

      Matrix Representation

      For finite sets A={a1,,am}A = \{a_1, \ldots, a_m\} and B={b1,,bn}B = \{b_1, \ldots, b_n\}, represent RR as an m×nm \times n boolean matrix MM:

      Mij=1    aiRbjM_{ij} = 1 \iff a_i R b_j

      Graph Representation (for endo-relations)

      Draw elements of AA as vertices. Draw a directed edge from aa to bb whenever aRbaRb.


      Summary

      • A relation R:ABR : A \nrightarrow B is a subset RA×BR \subseteq A \times B
      • Special relations: empty, full, identity
      • Key properties: reflexive, symmetric, transitive, antisymmetric
      • Visualise with internal diagrams, matrices, or directed graphs
    • Relational Composition

      Definition

      Given relations R:ABR : A \nrightarrow B and S:BCS : B \nrightarrow C, the composite SR:ACS \circ R : A \nrightarrow C is:

      SR={(a,c)A×C:bB.aRbbSc}S \circ R = \{(a, c) \in A \times C : \exists b \in B. aRb \land bSc\}

      In words: aa is related to cc by SRS \circ R iff there is some intermediate bb connecting them through RR and then SS.

      Notation Warning

      Some sources write composition as R;SR ; S or RSRS. The order convention varies. Our convention: (SR)(a)=S(R(a))(S \circ R)(a) = S(R(a)).

      Example

      Let R:{1,2,3}{a,b}R : \{1, 2, 3\} \nrightarrow \{a, b\} with R={(1,a),(2,b),(3,a)}R = \{(1, a), (2, b), (3, a)\}.

      Let S:{a,b}{X,Y}S : \{a, b\} \nrightarrow \{X, Y\} with S={(a,X),(b,Y),(b,X)}S = \{(a, X), (b, Y), (b, X)\}.

      Then SR={(1,X),(2,X),(2,Y),(3,X)}S \circ R = \{(1, X), (2, X), (2, Y), (3, X)\}.

      Check: 1Ra1Ra and aSXaSX, so (1,X)SR(1, X) \in S \circ R. And 2Rb2Rb with bSYbSY and bSXbSX, so (2,Y),(2,X)SR(2, Y), (2, X) \in S \circ R.

      Examinable Proof: Associativity

      Theorem: Relational composition is associative: (TS)R=T(SR)(T \circ S) \circ R = T \circ (S \circ R).

      Proof:

      By extensionality, we show both sides relate the same pairs.

      a((TS)R)d    c.aRcc(TS)da ((T \circ S) \circ R) d \iff \exists c. aRc \land c(T \circ S)d     c.aRc(b.cSbbTd)\iff \exists c. aRc \land (\exists b. cSb \land bTd)     b,c.aRccSbbTd\iff \exists b, c. aRc \land cSb \land bTd

      a(T(SR))d    b.a(SR)bbTda (T \circ (S \circ R)) d \iff \exists b. a(S \circ R)b \land bTd     b.(c.aRccSb)bTd\iff \exists b. (\exists c. aRc \land cSb) \land bTd     b,c.aRccSbbTd\iff \exists b, c. aRc \land cSb \land bTd

      Both sides are equivalent to b,c.aRccSbbTd\exists b, c. aRc \land cSb \land bTd.

      Therefore (TS)R=T(SR)(T \circ S) \circ R = T \circ (S \circ R).

      Examinable Proof: Identity Laws

      Theorem: The identity relation is a neutral element for composition.

      idBR=R=RidA\text{id}_B \circ R = R = R \circ \text{id}_A

      for R:ABR : A \nrightarrow B.

      Proof:

      a(idBR)b    cB.aRccidBba(\text{id}_B \circ R)b \iff \exists c \in B. aRc \land c\,\text{id}_B\,b     cB.aRcc=b\iff \exists c \in B. aRc \land c = b     aRb\iff aRb

      Similarly: a(RidA)b    cA.aidAccRba(R \circ \text{id}_A)b \iff \exists c \in A. a\,\text{id}_A\,c \land cRb     cA.a=ccRb\iff \exists c \in A. a = c \land cRb     aRb\iff aRb

      Monoid Structure

      The endo-relations Rel(A,A)\text{Rel}(A, A) on a set AA form a monoid under composition:

      • Associativity: (RS)T=R(ST)(R \circ S) \circ T = R \circ (S \circ T)
      • Identity: RidA=R=idARR \circ \text{id}_A = R = \text{id}_A \circ R

      This is a foundational structure with many applications.

      n-fold Iteration

      For a relation RR on AA, define the nn-fold iteration:

      R0=idAR^{\circ 0} = \text{id}_A R(n+1)=RRnR^{\circ (n+1)} = R \circ R^{\circ n}

      Interpretation

      • R0R^{\circ 0}: “be the same element”
      • R1=RR^{\circ 1} = R: “take one RR-step”
      • RnR^{\circ n}: “take exactly nn RR-steps”

      The Path Relation

      The path relation (or reflexive-transitive closure) is:

      R=n0RnR^{\circ *} = \bigcup_{n \geq 0} R^{\circ n}

      This relates aa to bb iff there is a path of length 0\geq 0 from aa to bb.

      Properties

      • Always reflexive: idAR\text{id}_A \subseteq R^{\circ *}
      • Always transitive: RR=RR^{\circ *} \circ R^{\circ *} = R^{\circ *}

      Summary

      • SR={(a,c):b.aRbbSc}S \circ R = \{(a, c) : \exists b. aRb \land bSc\}
      • Composition is associative with identity id\text{id}
      • Endo-relations form a monoid under composition
      • RnR^{\circ n} is nn-fold iteration; R=nRnR^{\circ *} = \bigcup_n R^{\circ n} is the path relation
    • Relations as Matrices

      Boolean Matrices

      A boolean matrix is a matrix whose entries are 00 or 11, with operations:

      • And (\land): for multiplication-like operations
      • Or (\lor): for addition-like operations

      The Boolean Semiring

      The boolean semiring B={0,1}\mathbb{B} = \{0, 1\} has:

      • Addition: 00=00 \lor 0 = 0, 01=10 \lor 1 = 1, 10=11 \lor 0 = 1, 11=11 \lor 1 = 1 (OR)
      • Multiplication: 00=00 \land 0 = 0, 01=00 \land 1 = 0, 10=01 \land 0 = 0, 11=11 \land 1 = 1 (AND)

      Matrix-Relation Correspondence

      From Relation to Matrix

      For finite sets A=[m]A = [m] and B=[n]B = [n], a relation R:ABR : A \nrightarrow B corresponds to an m×nm \times n boolean matrix mat(R)\text{mat}(R):

      mat(R)ij={1if iRj0otherwise\text{mat}(R)_{ij} = \begin{cases} 1 & \text{if } iRj \\ 0 & \text{otherwise} \end{cases}

      From Matrix to Relation

      Given an m×nm \times n boolean matrix MM, define rel(M):[m][n]\text{rel}(M) : [m] \nrightarrow [n] by:

      irel(M)j    Mij=1i\,\text{rel}(M)\,j \iff M_{ij} = 1

      Mutual Inverses

      rel(mat(R))=Randmat(rel(M))=M\text{rel}(\text{mat}(R)) = R \quad \text{and} \quad \text{mat}(\text{rel}(M)) = M

      This is a bijection between relations [m][n][m] \nrightarrow [n] and m×nm \times n boolean matrices.

      Matrix Operations for Relations

      Composition as Matrix Multiplication

      For R:[m][n]R : [m] \nrightarrow [n] and S:[n][p]S : [n] \nrightarrow [p]:

      mat(SR)=mat(S)mat(R)\text{mat}(S \circ R) = \text{mat}(S) \otimes \text{mat}(R)

      where \otimes is boolean matrix multiplication:

      (MN)ik=j=1n(MijNjk)(M \otimes N)_{ik} = \bigvee_{j=1}^{n} (M_{ij} \land N_{jk})

      Union as Matrix Addition

      mat(RS)=mat(R)mat(S)\text{mat}(R \cup S) = \text{mat}(R) \oplus \text{mat}(S)

      where \oplus is boolean matrix addition (pointwise OR):

      (MN)ij=MijNij(M \oplus N)_{ij} = M_{ij} \lor N_{ij}

      Example

      Let R:{1,2}{a,b}R : \{1, 2\} \nrightarrow \{a, b\} with R={(1,a),(2,b)}R = \{(1, a), (2, b)\}.

      mat(R)=(1001)\text{mat}(R) = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}

      Let S:{a,b}{X,Y}S : \{a, b\} \nrightarrow \{X, Y\} with S={(a,X),(b,Y)}S = \{(a, X), (b, Y)\}.

      mat(S)=(1001)\text{mat}(S) = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}

      Then SR={(1,X),(2,Y)}S \circ R = \{(1, X), (2, Y)\} and:

      mat(S)mat(R)=(1001)(1001)=(1001)\text{mat}(S) \otimes \text{mat}(R) = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \otimes \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}

      Reachability via Matrix Powers

      The Adjacency Matrix

      For a relation R:[n][n]R : [n] \nrightarrow [n], the matrix mat(R)\text{mat}(R) is the adjacency matrix of the corresponding directed graph.

      Path Matrices

      The matrix mat(R)k=mat(R)mat(R)\text{mat}(R)^{\circ k} = \text{mat}(R) \otimes \cdots \otimes \text{mat}(R) (k times) encodes paths of length exactly kk.

      Computing Reachability

      Define:

      M0=In(identity matrix)M_0 = I_n \quad \text{(identity matrix)} Mk+1=In(MMk)M_{k+1} = I_n \oplus (M \otimes M_k)

      Then Mn=mat(R)M_n = \text{mat}(R^{\circ *}), the adjacency matrix of the path relation.

      Why This Works

      • A simple path in an nn-vertex graph has length <n< n
      • MkM_k accumulates all paths of length k\leq k
      • By step nn, all reachable vertices are discovered

      Example

      For R={(1,2),(2,3),(3,1)}R = \{(1, 2), (2, 3), (3, 1)\} (a 3-cycle):

      mat(R)=(010001100)\text{mat}(R) = \begin{pmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \\ 1 & 0 & 0 \end{pmatrix}

      Computing M3M_3 gives mat(R)=mat({1,2,3}×{1,2,3})\text{mat}(R^{\circ *}) = \text{mat}(\{1, 2, 3\} \times \{1, 2, 3\}) (everything reachable).


      Semirings for Different Graph Problems

      The power of matrix multiplication is that changing the underlying semiring changes what problem we solve.

      The General Formula

      Matrix multiplication over a semiring (S,,,0,1)(S, \oplus, \otimes, 0, 1):

      (MN)ik=j=1n(MijNjk)(M \otimes N)_{ik} = \bigoplus_{j=1}^{n} (M_{ij} \otimes N_{jk})

      This formula works for ANY semiring!

      Comparison of Semirings

      SemiringAddition \oplusMultiply \otimesZeroOneComputes
      Boolean\lor (OR)\land (AND)0011Reachability
      Min-Plusmin\min++\infty00Shortest path
      Max-Plusmax\max++-\infty00Longest path
      Standard++×\times0011Path counting

      Semiring comparison for graph algorithms

      The Boolean Semiring: Reachability

      For adjacency matrix AA:

      Aijk=1    exists path of length k from i to jA^k_{ij} = 1 \iff \text{exists path of length } k \text{ from } i \text{ to } j

      Aij=1    exists ANY path from i to jA^*_{ij} = 1 \iff \text{exists ANY path from } i \text{ to } j

      The Min-Plus Semiring: Shortest Path

      For weighted adjacency matrix WW (where WijW_{ij} = weight of edge iji \to j, \infty if no edge):

      Wij(k)=minj1,,jk1(Wij1+Wj1j2++Wjk1j)W_{ij}^{(k)} = \min_{j_1, \ldots, j_{k-1}}(W_{ij_1} + W_{j_1 j_2} + \cdots + W_{j_{k-1}j})

      This is the minimum cost of any path of exactly kk edges.

      Wij=shortest path cost from i to jW^*_{ij} = \text{shortest path cost from } i \text{ to } j

      Example: Shortest Path

      Consider weighted graph:

          1 --3--> 2
          |        |
          4        1
          |        |
          v        v
          3 --2--> 4

      Adjacency matrix (min-plus):

      W=(3412)W = \begin{pmatrix} \infty & 3 & 4 & \infty \\ \infty & \infty & \infty & 1 \\ \infty & \infty & \infty & 2 \\ \infty & \infty & \infty & \infty \end{pmatrix}

      W2W^2 computes shortest paths of exactly 2 edges:

      W1,42=min(W1,2+W2,4,W1,3+W3,4)=min(3+1,4+2)=4W^2_{1,4} = \min(W_{1,2} + W_{2,4}, W_{1,3} + W_{3,4}) = \min(3+1, 4+2) = 4

      The path 1241 \to 2 \to 4 costs 4, which is cheaper than 1341 \to 3 \to 4 (cost 6).

      Identity Elements in Min-Plus

      • Additive identity (\oplus-identity): \infty (since min(x,)=x\min(x, \infty) = x)
      • Multiplicative identity (\otimes-identity): 00 (since x+0=xx + 0 = x)

      Annnihilation Property

      +x=\infty + x = \infty

      Even if a path exists (xx finite), if you must traverse a non-existent edge (\infty), the total is impossible.

      Warning: Negative Weights

      The min-plus semiring works correctly only when:

      • All edge weights are non-negative, OR
      • The graph has no negative-weight cycles

      If negative cycles exist, shortest paths may not be well-defined.


      Summary

      • Relations on finite sets correspond to boolean matrices
      • Composition corresponds to boolean matrix multiplication
      • Union corresponds to boolean matrix addition
      • Matrix powers compute reachability in directed graphs
      • Changing the semiring changes the problem: Boolean = reachability, Min-Plus = shortest path
      • Min-Plus semiring: =min\oplus = \min, =+\otimes = +, identities are \infty and 00
    • Directed Graphs, Paths, and Reachability

      Directed Graphs

      Definition

      A directed graph (or digraph) is a pair (V,E)(V, E) where:

      • VV is a set of vertices (or nodes)
      • EV×VE \subseteq V \times V is a set of edges (or arcs)

      Note: A digraph is exactly an endo-relation E:VVE : V \nrightarrow V.

      Visualisation

      Draw each vertex as a point. Draw an arrow from vv to ww iff (v,w)E(v, w) \in E.

      A directed graph with 4 vertices

      Paths

      Definition

      A path (or walk) in a digraph is a sequence v0,v1,,vkv_0, v_1, \ldots, v_k such that (vi1,vi)E(v_{i-1}, v_i) \in E for each i=1,,ki = 1, \ldots, k.

      • The length of the path is kk (the number of edges)
      • A simple path visits no vertex twice
      • A cycle is a path from a vertex back to itself

      Path Relation

      The path relation EE^{\circ *} relates vv to ww iff there exists a path from vv to ww:

      E=n0EnE^{\circ *} = \bigcup_{n \geq 0} E^{\circ n}

      where EnE^{\circ n} is the nn-fold composite.

      Reachability

      Definition

      Vertex ww is reachable from vertex vv if there is a path from vv to ww.

      Equivalently: vEwv E^{\circ *} w.

      Computing Reachability

      For an nn-vertex graph, any simple path has length <n< n. Therefore:

      E=k=0n1EkE^{\circ *} = \bigcup_{k=0}^{n-1} E^{\circ k}

      This gives an algorithm: compute IEE2En1I \cup E \cup E^2 \cup \cdots \cup E^{n-1}.

      Adjacency Matrix Approach

      Let MM be the adjacency matrix. Compute:

      M=IMM2Mn1M^* = I \oplus M \oplus M^2 \oplus \cdots \oplus M^{n-1}

      Then Mij=1M^*_{ij} = 1 iff jj is reachable from ii.

      Preorders

      Definition

      A relation RR on AA is a preorder if it is:

      1. Reflexive: a.aRa\forall a. aRa
      2. Transitive: a,b,c.(aRbbRcaRc)\forall a, b, c. (aRb \land bRc \Rightarrow aRc)

      A preorder is a partial order if it is also antisymmetric: (aRbbRa)a=b(aRb \land bRa) \Rightarrow a = b.

      Examples

      RelationType
      \leq on R\mathbb{R}Partial order
      \subseteq on P(X)\mathcal{P}(X)Partial order
      \mid (divisibility) on Z+\mathbb{Z}^+Partial order
      \mid on Z\mathbb{Z}Preorder (not antisymmetric: 11-1 \mid 1 and 111 \mid -1)
      Reachability in a digraphPreorder

      Reflexive-Transitive Closure

      Definition

      The reflexive-transitive closure of RR is the smallest preorder containing RR:

      R={P:RP,P is a preorder}R^{\circ *} = \bigcap \{P : R \subseteq P, P \text{ is a preorder}\}

      Examinable Proof: Characterisation

      Theorem: RR^{\circ *} is the smallest preorder containing RR.

      Proof:

      Part 1: RR^{\circ *} is a preorder containing RR.

      • RRR \subseteq R^{\circ *}: R1=RR^{\circ 1} = R, and R=n0RnR^{\circ *} = \bigcup_{n \geq 0} R^{\circ n}.
      • Reflexivity: R0=idRR^{\circ 0} = \text{id} \subseteq R^{\circ *}.
      • Transitivity: If aRba R^{\circ *} b and bRcb R^{\circ *} c, there exist paths from aa to bb and bb to cc. Concatenating gives a path from aa to cc.

      Part 2: RR^{\circ *} is the smallest such preorder.

      Let PP be any preorder with RPR \subseteq P. We show RPR^{\circ *} \subseteq P.

      By induction on nn: RnPR^{\circ n} \subseteq P for all nn.

      • Base (n=0n = 0): R0=idPR^{\circ 0} = \text{id} \subseteq P by reflexivity.
      • Step: R(n+1)=RRnPP=PR^{\circ (n+1)} = R \circ R^{\circ n} \subseteq P \circ P = P by transitivity.

      Thus nRnP\bigcup_n R^{\circ n} \subseteq P.

      Transitive Closure

      The transitive closure R+=n1RnR^{\circ +} = \bigcup_{n \geq 1} R^{\circ n} is the smallest transitive relation containing RR.


      Summary

      • A digraph (V,E)(V, E) is an endo-relation E:VVE : V \nrightarrow V
      • Path relation EE^{\circ *} relates connected vertices
      • RR^{\circ *} is the reflexive-transitive closure: smallest preorder containing RR
      • Reachability computed via matrix powers MM^* up to Mn1M^{n-1}
    • Preorders and Reflexive-Transitive Closure

      Preorders

      Definition

      A relation RR on AA is a preorder if it is:

      1. Reflexive: aA.aRa\forall a \in A. aRa
      2. Transitive: a,b,cA.(aRbbRcaRc)\forall a, b, c \in A. (aRb \land bRc \Rightarrow aRc)

      A preorder captures a notion of “being related through a chain of steps” with the base case of being related to oneself.

      Examples

      • \leq on R\mathbb{R}: reflexive and transitive
      • \subseteq on P(X)\mathcal{P}(X): reflexive and transitive
      • Reachability in a directed graph: always a preorder
      • Divisibility \mid on Z+\mathbb{Z}^+: reflexive and transitive

      Partial Orders

      Definition

      A partial order is a preorder that is also antisymmetric:

      aRbbRaa=baRb \land bRa \Rightarrow a = b

      Preorder vs Partial Order

      A preorder may have distinct elements aba \neq b with aRbaRb and bRabRa.

      Example: ”xx and yy live in the same country” is a preorder but not a partial order.

      Example: Divisibility on Z\mathbb{Z} is a preorder: 11-1 \mid 1 and 111 \mid -1, but 11-1 \neq 1.

      Quotient to Partial Order

      Every preorder RR induces an equivalence relation:

      ab    aRbbRaa \sim b \iff aRb \land bRa

      On the quotient A/A / {\sim}, the induced relation is a partial order.

      Operations on Preorders

      Intersection

      If RR and SS are preorders on AA, then RSR \cap S is a preorder.

      Proof: Reflexive since aRaaRa and aSaaSa, so a(RS)aa(R \cap S)a. Transitive since if a(RS)ba(R \cap S)b and b(RS)cb(R \cap S)c, then aRbaSbaRb \land aSb and bRcbScbRc \land bSc, so aRcaScaRc \land aSc.

      Composition

      If RR and SS are preorders, RSR \circ S need not be a preorder.

      Example: Let R=idR = \text{id} and SS be any preorder. Then RS=SR \circ S = S is a preorder. But for other relations, composition can fail transitivity.

      Reflexive-Transitive Closure

      Definition

      For any relation RR on AA, the reflexive-transitive closure is:

      R=n0RnR^{\circ *} = \bigcup_{n \geq 0} R^{\circ n}

      where R0=idAR^{\circ 0} = \text{id}_A and R(n+1)=RRnR^{\circ (n+1)} = R \circ R^{\circ n}.

      Alternative Definition

      R={P:RP,P is a preorder}R^{\circ *} = \bigcap \{P : R \subseteq P, P \text{ is a preorder}\}

      This is the intersection of all preorders containing RR.

      Examinable Proof: Equivalence of Definitions

      Theorem: The two definitions of RR^{\circ *} coincide.

      Proof:

      Step 1: RR^{\circ *} as defined by union is a preorder containing RR.

      • RRR \subseteq R^{\circ *}: Clear since R1=RR^{\circ 1} = R.
      • Reflexive: R0=idRR^{\circ 0} = \text{id} \subseteq R^{\circ *}.
      • Transitive: If aRba R^{\circ *} b and bRcb R^{\circ *} c, there exist m,nm, n with aRmba R^{\circ m} b and bRncb R^{\circ n} c. Then aR(m+n)ca R^{\circ (m+n)} c.

      Step 2: RR^{\circ *} is contained in every preorder containing RR.

      Let PP be a preorder with RPR \subseteq P. By induction, RnPR^{\circ n} \subseteq P for all nn.

      • Base: R0=idPR^{\circ 0} = \text{id} \subseteq P by reflexivity.
      • Step: R(n+1)=RRnPPPR^{\circ (n+1)} = R \circ R^{\circ n} \subseteq P \circ P \subseteq P by transitivity.

      Thus RPR^{\circ *} \subseteq P.

      Step 3: Since RR^{\circ *} is a preorder containing RR and is contained in all such preorders, it equals their intersection.

      Transitive Closure

      The transitive closure R+R^{\circ +} is defined similarly but without reflexivity:

      R+=n1RnR^{\circ +} = \bigcup_{n \geq 1} R^{\circ n}

      This is the smallest transitive relation containing RR.

      Relationship

      R=idR+R^{\circ *} = \text{id} \cup R^{\circ +}

      Computing Closure via Matrix

      For an nn-vertex graph with adjacency matrix MM:

      M=IMM2M(n1)M^* = I \oplus M \oplus M^{\otimes 2} \oplus \cdots \oplus M^{\otimes (n-1)}

      This computes RR^{\circ *} efficiently.


      Summary

      • A preorder is reflexive and transitive
      • A partial order is a preorder that is also antisymmetric
      • R=n0RnR^{\circ *} = \bigcup_{n \geq 0} R^{\circ n} is the reflexive-transitive closure
      • RR^{\circ *} is the smallest preorder containing RR
    • Functions: Partial and Total

      Partial Functions

      Definition

      A relation f:ABf : A \nrightarrow B is functional (or a partial function) if:

      aA.b1,b2B.(afb1afb2b1=b2)\forall a \in A. \forall b_1, b_2 \in B. (afb_1 \land afb_2 \Rightarrow b_1 = b_2)

      Each input aa is related to at most one output.

      Notation

      • f(a)f(a)\downarrow means ff is defined at aa (there exists bb with afbafb)
      • f(a)f(a)\uparrow means ff is undefined at aa
      • When defined, f(a)f(a) denotes the unique bb with afbafb

      Domain

      The domain of partial function f:ABf : A \nrightarrow B is:

      dom(f)={aA:f(a)}\text{dom}(f) = \{a \in A : f(a)\downarrow\}

      Total Functions

      Definition

      A partial function f:ABf : A \nrightarrow B is total if dom(f)=A\text{dom}(f) = A.

      We write f:ABf : A \to B for a total function from AA to BB.

      Verification: For all aAa \in A, there exists exactly one bBb \in B with afbafb.

      Examples

      FunctionTotal?Notes
      f:RR,f(x)=x2f : \mathbb{R} \to \mathbb{R}, f(x) = x^2YesDefined everywhere
      f:RR,f(x)=xf : \mathbb{R} \nrightarrow \mathbb{R}, f(x) = \sqrt{x}PartialOnly defined for x0x \geq 0
      f:RR,f(x)=1/xf : \mathbb{R} \nrightarrow \mathbb{R}, f(x) = 1/xPartialUndefined at x=0x = 0

      Cardinality of Function Spaces

      Partial Functions

      AB=(B+1)A|A \nrightarrow B| = (|B| + 1)^{|A|}

      Each of the A|A| inputs independently chooses one of B|B| outputs or “undefined”.

      Total Functions

      AB=BA|A \to B| = |B|^{|A|}

      Each of the A|A| inputs independently chooses one of B|B| outputs.

      Example

      For A={0,1}A = \{0, 1\} and B={a,b}B = \{a, b\}:

      • AB=22=4|A \to B| = 2^2 = 4: constant functions xax \mapsto a, xbx \mapsto b, swap 0a,1b0 \mapsto a, 1 \mapsto b, swap 0b,1a0 \mapsto b, 1 \mapsto a
      • AB=(2+1)2=9|A \nrightarrow B| = (2+1)^2 = 9: the 4 total functions plus 5 partial ones

      Composition of Functions

      Partial Function Composition

      Given f:ABf : A \nrightarrow B and g:BCg : B \nrightarrow C, define gf:ACg \circ f : A \nrightarrow C by:

      (gf)(a)={g(f(a))if f(a) and g(f(a))undefinedotherwise(g \circ f)(a) = \begin{cases} g(f(a)) & \text{if } f(a)\downarrow \text{ and } g(f(a))\downarrow \\ \text{undefined} & \text{otherwise} \end{cases}

      Total Function Composition

      Given f:ABf : A \to B and g:BCg : B \to C, the composite gf:ACg \circ f : A \to C is always total:

      aA.(gf)(a)=g(f(a))\forall a \in A. (g \circ f)(a) = g(f(a))

      Closure Properties

      • The composite of two functional relations is functional
      • The composite of two total functions is total

      Identity Function

      The identity function idA:AA\text{id}_A : A \to A is:

      idA(a)=afor all aA\text{id}_A(a) = a \quad \text{for all } a \in A

      This is total and serves as the unit for composition.

      Function Application vs Relational Image

      For a function f:ABf : A \to B and SAS \subseteq A:

      • Direct image: f(S)={f(a):aS}Bf(S) = \{f(a) : a \in S\} \subseteq B
      • Inverse image: f1(T)={a:f(a)T}Af^{-1}(T) = \{a : f(a) \in T\} \subseteq A

      These extend relational images to the functional case.


      Summary

      • A partial function f:ABf : A \nrightarrow B maps each aa to at most one bb
      • A total function f:ABf : A \to B maps each aa to exactly one bb
      • AB=BA|A \to B| = |B|^{|A|} and AB=(B+1)A|A \nrightarrow B| = (|B|+1)^{|A|}
      • Composition preserves functionality; total functions are closed under composition
    • Bijections and Indicator Functions

      Surjections

      Definition

      A function f:ABf : A \to B is surjective (or onto) if:

      bB.aA.f(a)=b\forall b \in B. \exists a \in A. f(a) = b

      Every element of BB is the image of some element of AA.

      Notation

      Write f:ABf : A \twoheadrightarrow B for a surjection.

      Examples

      • f:RR0,f(x)=x2f : \mathbb{R} \to \mathbb{R}_{\geq 0}, f(x) = x^2 is surjective
      • f:ZZ,f(n)=2nf : \mathbb{Z} \to \mathbb{Z}, f(n) = 2n is not surjective (odd numbers not in image)

      Injections

      Definition

      A function f:ABf : A \to B is injective (or one-to-one) if:

      a1,a2A.(f(a1)=f(a2)a1=a2)\forall a_1, a_2 \in A. (f(a_1) = f(a_2) \Rightarrow a_1 = a_2)

      Equivalently: a1a2f(a1)f(a2)a_1 \neq a_2 \Rightarrow f(a_1) \neq f(a_2).

      Notation

      Write f:ABf : A \hookrightarrow B or f:ABf : A \rightarrowtail B for an injection.

      Examples

      • f:ZZ,f(n)=2nf : \mathbb{Z} \to \mathbb{Z}, f(n) = 2n is injective
      • f:RR,f(x)=x2f : \mathbb{R} \to \mathbb{R}, f(x) = x^2 is not injective (f(1)=f(1)f(1) = f(-1))

      Bijections

      Definition

      A function is a bijection if it is both injective and surjective.

      Equivalently: for all bBb \in B, there exists a unique aAa \in A with f(a)=bf(a) = b.

      Inverse Function

      If f:ABf : A \to B is bijective, there exists a unique f1:BAf^{-1} : B \to A such that:

      f1(f(a))=aandf(f1(b))=bf^{-1}(f(a)) = a \quad \text{and} \quad f(f^{-1}(b)) = b

      Notation

      Write f:ABf : A \cong B or ABA \sim B to indicate a bijection exists between AA and BB.

      Counting Bijections

      For finite sets A,BA, B:

      Iso(A,B)={0ABn!A=B=n|\text{Iso}(A, B)| = \begin{cases} 0 & |A| \neq |B| \\ n! & |A| = |B| = n \end{cases}

      Reason: Build a bijection by successively assigning images. Round kk has nkn - k remaining choices.

      Examinable Proof: Isomorphisms are Invertible

      Theorem: For f:ABf : A \to B, TFAE:

      1. ff is invertible (there exists g:BAg : B \to A with gf=idAg \circ f = \text{id}_A and fg=idBf \circ g = \text{id}_B)
      2. bB.!aA.f(a)=b\forall b \in B. \exists! a \in A. f(a) = b
      3. ff is bijective

      Proof sketch:

      (12)(1 \Rightarrow 2): If gg is the inverse, take a=g(b)a = g(b). Uniqueness: if f(a1)=f(a2)=bf(a_1) = f(a_2) = b, then a1=g(f(a1))=g(b)=g(f(a2))=a2a_1 = g(f(a_1)) = g(b) = g(f(a_2)) = a_2.

      (23)(2 \Rightarrow 3): Existence gives surjectivity; uniqueness gives injectivity.

      (31)(3 \Rightarrow 1): Define f1(b)f^{-1}(b) as the unique aa with f(a)=bf(a) = b. Verify both compositions give identity.

      Indicator Functions

      Definition

      For SAS \subseteq A, the indicator function (or characteristic function) χS:A{0,1}\chi_S : A \to \{0, 1\} is:

      χS(a)={1aS0aS\chi_S(a) = \begin{cases} 1 & a \in S \\ 0 & a \notin S \end{cases}

      Comprehension

      Given ϕ:A{0,1}\phi : A \to \{0, 1\}, the comprehension is:

      [ϕ]={aA:ϕ(a)=1}[\phi] = \{a \in A : \phi(a) = 1\}

      Bijection

      P(A)(A{0,1})\mathcal{P}(A) \cong (A \to \{0, 1\})

      Every subset corresponds to a unique indicator function, and vice versa.

      Finite Cardinality

      Definition

      A set AA is finite if there exists nNn \in \mathbb{N} and a bijection A[n]={0,1,,n1}A \cong [n] = \{0, 1, \ldots, n-1\}.

      The cardinality of AA, written A|A|, is this unique nn.

      Isomorphism as Equality of Cardinality

      For finite sets: AB    A=BA \cong B \iff |A| = |B|.


      Summary

      • Surjection: every bb is hit; injection: distinct inputs give distinct outputs
      • Bijection = invertible function
      • Iso(A,B)=n!|\text{Iso}(A, B)| = n! if A=B=n|A| = |B| = n
      • Indicator functions establish P(A)(A{0,1})\mathcal{P}(A) \cong (A \to \{0, 1\})
    • Equivalence Relations and Partitions

      Equivalence Relations

      Definition

      A relation EE on AA is an equivalence relation if it is:

      1. Reflexive: aA.aEa\forall a \in A. aEa
      2. Symmetric: a,bA.(aEbbEa)\forall a, b \in A. (aEb \Rightarrow bEa)
      3. Transitive: a,b,cA.(aEbbEcaEc)\forall a, b, c \in A. (aEb \land bEc \Rightarrow aEc)

      Equivalence Classes

      The equivalence class of aa under EE is:

      [a]E={bA:aEb}[a]_E = \{b \in A : aEb\}

      Examples

      RelationEquivalence Classes
      a=ba = b on any setSingletons {a}\{a\} for each aa
      Congruence mod nn{0,n,2n,},{1,n+1,},\{0, n, 2n, \ldots\}, \{1, n+1, \ldots\}, \ldots
      “Same parity” on Z\mathbb{Z}{even integers}, {odd integers}
      Universal relation A×AA \times ASingle class containing all of AA

      Partitions

      Definition

      A partition PP of AA is a collection of non-empty, pairwise disjoint subsets whose union is AA:

      1. BP.B\forall B \in P. B \neq \emptyset
      2. B1,B2P.(B1B2B1B2=)\forall B_1, B_2 \in P. (B_1 \neq B_2 \Rightarrow B_1 \cap B_2 = \emptyset)
      3. BPB=A\bigcup_{B \in P} B = A

      The elements of a partition are called blocks.

      Examples

      • {{1,2},{3},{4,5,6}}\{\{1, 2\}, \{3\}, \{4, 5, 6\}\} is a partition of {1,2,3,4,5,6}\{1, 2, 3, 4, 5, 6\}
      • {{1},{2},{3}}\{\{1\}, \{2\}, \{3\}\} is a partition of {1,2,3}\{1, 2, 3\}
      • {{evens},{odds}}\{\{\text{evens}\}, \{\text{odds}\}\} is a partition of Z\mathbb{Z}

      Examinable Proof: Bijection between Equivalence Relations and Partitions

      Theorem: There is a natural bijection Φ:EqRel(A)Part(A)\Phi : \text{EqRel}(A) \cong \text{Part}(A).

      From Equivalence Relation to Partition

      Given an equivalence relation EE, define:

      Φ(E)={[a]E:aA}\Phi(E) = \{[a]_E : a \in A\}

      Well-defined: The equivalence classes form a partition.

      • Non-empty: a[a]Ea \in [a]_E by reflexivity
      • Disjoint: If b[a]E[a]Eb \in [a]_E \cap [a']_E, then aEbaEb and aEba'Eb. By symmetry and transitivity, aEaaEa', so [a]E=[a]E[a]_E = [a']_E.
      • Covering: Every aa is in its own class

      From Partition to Equivalence Relation

      Given a partition PP, define:

      Φ1(P)={(a,b):BP.aBbB}\Phi^{-1}(P) = \{(a, b) : \exists B \in P. a \in B \land b \in B\}

      Well-defined: This is an equivalence relation.

      • Reflexive: aa is in exactly one block BB, so (a,a)Φ1(P)(a, a) \in \Phi^{-1}(P)
      • Symmetric: If a,bBa, b \in B, then b,aBb, a \in B
      • Transitive: If a,bBa, b \in B and b,cBb, c \in B', then bBBb \in B \cap B'. Since blocks are disjoint, B=BB = B', so a,cBa, c \in B.

      Inverses

      Φ1(Φ(E))=E\Phi^{-1}(\Phi(E)) = E: aΦ1(Φ(E))ba \Phi^{-1}(\Phi(E)) b iff a,ba, b are in the same equivalence class of EE, iff aEbaEb.

      Φ(Φ1(P))=P\Phi(\Phi^{-1}(P)) = P: The blocks of Φ(Φ1(P))\Phi(\Phi^{-1}(P)) are the equivalence classes under “in same block of PP”, which are the blocks themselves.

      Key Step Explanation

      The single crucial fact: every aAa \in A lies in exactly one block of PP.

      • No fewer (blocks cover AA)
      • No more (blocks are disjoint)

      This drives both directions of the proof.

      Quotient Set

      The quotient A/EA / E is the set of equivalence classes:

      A/E={[a]E:aA}A / E = \{[a]_E : a \in A\}

      Canonical Map

      π:AA/E\pi : A \to A/E sends aa to [a]E[a]_E.

      This is the universal surjection associated with EE.


      Summary

      • Equivalence relations: reflexive, symmetric, transitive
      • Partitions: non-empty, disjoint blocks covering AA
      • Bijection: E{[a]E}E \mapsto \{[a]_E\} and P{(a,b):a,b in same block}P \mapsto \{(a, b) : a, b \text{ in same block}\}
      • Quotient A/EA/E is the set of equivalence classes
  • Cardinality and Infinite Sets

    Sections/retractions, enumerability, injectivity/countability, Cantor's theorem, and well-foundedness

    • Sections, Retractions, and Idempotents

      Section-Retraction Pairs

      Definition

      A section-retraction pair is a pair of functions s:BAs : B \to A and r:ABr : A \to B such that:

      rs=idBr \circ s = \text{id}_B

      • ss is called the section (or left inverse)
      • rr is called the retraction (or right inverse)

      Intuition

      • ss embeds BB into AA
      • rr projects back from AA to BB
      • Composing rr after ss returns to where we started in BB

      Section-retraction pair diagram

      Examples

      • BABB \hookrightarrow A \twoheadrightarrow B where ss is inclusion and rr is identity on BB
      • s:NZs : \mathbb{N} \to \mathbb{Z}, s(n)=ns(n) = n; r:ZNr : \mathbb{Z} \to \mathbb{N}, r(n)=nr(n) = |n| fails since r(s(1))r(s(-1)) is undefined
      • Correct: s:NZs : \mathbb{N} \to \mathbb{Z}, s(n)=ns(n) = n; r:ZNr : \mathbb{Z} \to \mathbb{N}, r(n)=max(n,0)r(n) = \max(n, 0)

      Properties

      Section is Injective

      If ss is a section of rr, then ss is injective.

      Proof: If s(b1)=s(b2)s(b_1) = s(b_2), then b1=r(s(b1))=r(s(b2))=b2b_1 = r(s(b_1)) = r(s(b_2)) = b_2.

      Retraction is Surjective

      If rr is a retraction of ss, then rr is surjective.

      Proof: For any bBb \in B, b=r(s(b))b = r(s(b)), so bb is in the image of rr.

      Converse Requires Choice

      If s:BAs : B \to A is injective, there exists a retraction r:ABr : A \to B (on image of ss) iff we can “pick” values for elements outside the image.

      This requires the Axiom of Choice in general.

      Idempotents

      Definition

      A function e:AAe : A \to A is idempotent if ee=ee \circ e = e.

      Intuition

      Applying ee once or twice gives the same result. ee “freezes” elements to a fixed form.

      Examples

      • Projection (x,y)(x,0)(x, y) \mapsto (x, 0) is idempotent
      • Identity is idempotent
      • Constant function xcx \mapsto c is idempotent iff c=cc = c (always true)

      Examinable Proof: Idempotents and Section-Retraction Pairs

      Theorem: There is a bijection between:

      • Section-retraction pairs (s,r)(s, r) with s:BAs : B \to A, r:ABr : A \to B, rs=idBr \circ s = \text{id}_B
      • Idempotents e:AAe : A \to A

      From Section-Retraction to Idempotent

      Given (s,r)(s, r) with rs=idBr \circ s = \text{id}_B, define e=sr:AAe = s \circ r : A \to A.

      Idempotence:

      ee=srsr=sidBr=sr=ee \circ e = s \circ r \circ s \circ r = s \circ \text{id}_B \circ r = s \circ r = e

      From Idempotent to Section-Retraction

      Given idempotent e:AAe : A \to A, define:

      • B=Im(e)={e(a):aA}B = \text{Im}(e) = \{e(a) : a \in A\}
      • r:ABr : A \to B by r(a)=e(a)r(a) = e(a)
      • s:BAs : B \hookrightarrow A the inclusion

      Verification: r(s(b))=r(b)=e(b)r(s(b)) = r(b) = e(b).

      If bBb \in B, then b=e(a)b = e(a) for some aa. So e(b)=e(e(a))=e(a)=be(b) = e(e(a)) = e(a) = b by idempotence.

      Thus rs=idBr \circ s = \text{id}_B.

      These are Inverses

      Starting with (s,r)(s, r): the associated idempotent srs \circ r has image isomorphic to BB (via ss).

      Starting with ee: the section-retraction pair recovers ee as srs \circ r.

      Splitting Idempotents

      The process of going from ee to (s,r)(s, r) is called splitting the idempotent.

      The set B=Im(e)B = \text{Im}(e) is the “fixed points” of ee, and ee is the identity on BB.


      Summary

      • Section-retraction: rs=idBr \circ s = \text{id}_B
      • Sections are injective; retractions are surjective
      • Idempotent: ee=ee \circ e = e
      • Bijection: (s,r)sr(s, r) \mapsto s \circ r and e(inclusion,eIm(e))e \mapsto (\text{inclusion}, e|_{\text{Im}(e)})
    • Isomorphisms

      Definition

      A function f:ABf : A \to B is an isomorphism if there exists g:BAg : B \to A with:

      gf=idAandfg=idBg \circ f = \text{id}_A \quad \text{and} \quad f \circ g = \text{id}_B

      We write f:ABf : A \cong B or ABA \cong B.

      Uniqueness of Inverse

      If ff is an isomorphism, its inverse gg is unique, denoted f1f^{-1}.

      Proof: If g1g_1 and g2g_2 are both inverses, then g1=g1idB=g1fg2=idAg2=g2g_1 = g_1 \circ \text{id}_B = g_1 \circ f \circ g_2 = \text{id}_A \circ g_2 = g_2.

      Isomorphism is Equivalence

      Reflexivity

      idA:AA\text{id}_A : A \cong A with inverse idA\text{id}_A.

      Symmetry

      If f:ABf : A \cong B with inverse gg, then g:BAg : B \cong A with inverse ff.

      Transitivity

      If f:ABf : A \cong B and g:BCg : B \cong C, then gf:ACg \circ f : A \cong C.

      Proof: (gf)1=f1g1(g \circ f)^{-1} = f^{-1} \circ g^{-1}, since (gf)(f1g1)=gidBg1=idC(g \circ f) \circ (f^{-1} \circ g^{-1}) = g \circ \text{id}_B \circ g^{-1} = \text{id}_C.

      Isomorphism via Injection and Surjection

      Examinable Proof

      Theorem: For f:ABf : A \to B, TFAE:

      1. ff is an isomorphism
      2. ff is both injective and surjective
      3. bB.!aA.f(a)=b\forall b \in B. \exists! a \in A. f(a) = b

      Proof:

      (12)(1 \Rightarrow 2): If ff has inverse gg, then:

      • Injective: f(a1)=f(a2)g(f(a1))=g(f(a2))a1=a2f(a_1) = f(a_2) \Rightarrow g(f(a_1)) = g(f(a_2)) \Rightarrow a_1 = a_2.
      • Surjective: For any bb, f(g(b))=bf(g(b)) = b, so bb is in the image.

      (23)(2 \Rightarrow 3): Surjectivity gives existence; injectivity gives uniqueness.

      (31)(3 \Rightarrow 1): Define f1(b)=f^{-1}(b) = the unique aa with f(a)=bf(a) = b. Verify f1(f(a))=af^{-1}(f(a)) = a and f(f1(b))=bf(f^{-1}(b)) = b.

      Counting Isomorphisms

      For finite sets with A=B=n|A| = |B| = n:

      Iso(A,B)=n!|\text{Iso}(A, B)| = n!

      Combinatorial Argument

      Build a bijection by successively assigning images:

      • f(a1)f(a_1): nn choices
      • f(a2)f(a_2): n1n-1 remaining choices
      • f(an)f(a_n): 1 remaining choice

      Product: n(n1)1=n!n \cdot (n-1) \cdot \ldots \cdot 1 = n!

      Useful Isomorphisms

      Cartesian Product

      [m]×[n][mn][m] \times [n] \cong [mn]

      Via f(i,j)=ni+jf(i, j) = n \cdot i + j (row-major encoding).

      Disjoint Union

      [m]+[n][m+n][m] + [n] \cong [m+n]

      Via f(0,i)=if(0, i) = i and f(1,j)=m+jf(1, j) = m + j.

      Powerset

      P(X)(X{0,1})\mathcal{P}(X) \cong (X \to \{0, 1\})

      Via indicator functions.

      Powerset with One Extra Element

      P(X+[1])P(X)+P(X)\mathcal{P}(X + [1]) \cong \mathcal{P}(X) + \mathcal{P}(X)

      Reason: A subset of X+{}X + \{*\} either does or does not contain *, giving two copies of subsets of XX.

      Finite Cardinality

      For finite sets, ABA \cong B means A=B|A| = |B|.

      The combinatorial results above verify:

      • A×B=AB|A \times B| = |A| \cdot |B|
      • A+B=A+B|A + B| = |A| + |B|
      • P(A)=2A|\mathcal{P}(A)| = 2^{|A|}

      Summary

      • Isomorphism = invertible function = bijection
      • \cong is an equivalence relation on sets
      • Iso(A,B)=n!|\text{Iso}(A, B)| = n! if A=B=n|A| = |B| = n
      • Key isomorphisms: products, disjoint unions, powersets
    • Surjections and Enumerability

      Surjections

      Definition

      A function f:ABf : A \to B is surjective if:

      bB.aA.f(a)=b\forall b \in B. \exists a \in A. f(a) = b

      Every element of BB is “hit” by ff.

      Notation

      Write f:ABf : A \twoheadrightarrow B for a surjection.

      Enumerability

      Definition

      A set AA is enumerable if there exists a surjection NA\mathbb{N} \twoheadrightarrow A.

      A set is countable if it is finite or enumerable.

      Intuition

      An enumerable set can be “listed” in some order (possibly with repetition, possibly never ending).

      The surjection e:NAe : \mathbb{N} \twoheadrightarrow A gives an enumeration: e(0),e(1),e(2),e(0), e(1), e(2), \ldots

      Examples

      • N\mathbb{N}: enumerated by identity function
      • Z\mathbb{Z}: enumerated by e(0)=0e(0) = 0, e(2n+1)=n+1e(2n+1) = n+1, e(2n)=ne(2n) = -n for n>0n > 0
        • Enumeration: 0,1,1,2,2,3,3,0, -1, 1, -2, 2, -3, 3, \ldots
      • N×N\mathbb{N} \times \mathbb{N}: enumerated by Cantor pairing

      Closure Properties

      Subsets of Enumerable Sets

      If AA is enumerable and BAB \subseteq A, then BB is countable.

      Proof sketch: Let e:NAe : \mathbb{N} \twoheadrightarrow A. Define e:NBe' : \mathbb{N} \twoheadrightarrow B by skipping elements not in BB.

      If B=B = \emptyset, it is finite (countable). If BB \neq \emptyset, enumerate by visiting each e(n)e(n) in order and including it if in BB.

      Sum of Countable Sets

      If AA and BB are countable, then A+BA + B is countable.

      Proof: Interleave the enumerations.

      Product of Countable Sets

      If AA and BB are countable, then A×BA \times B is countable.

      Proof: Given enumerations eAe_A, eBe_B, enumerate pairs via a zigzag path through N×N\mathbb{N} \times \mathbb{N}.

      Cantor zigzag enumeration of N times N

      The Axiom of Choice

      Statement

      Every surjection has a section.

      f:AB.s:BA.fs=idB\forall f : A \twoheadrightarrow B. \exists s : B \hookrightarrow A. f \circ s = \text{id}_B

      Equivalence

      TFAE:

      1. Axiom of Choice (as above)
      2. Every family of non-empty sets has a choice function
      3. Every set can be well-ordered

      For Countable Sets

      For countable sets, enumerability provides an explicit section: given e:NAe : \mathbb{N} \twoheadrightarrow A and f:ABf : A \twoheadrightarrow B, define s(b)=e(min{n:f(e(n))=b})s(b) = e(\min\{n : f(e(n)) = b\}).

      This is the canonical witness, avoiding AC.

      Enumerability of Z\mathbb{Z}

      Explicit Bijection

      Define e:NZe : \mathbb{N} \to \mathbb{Z}:

      e(n)={0n=0n+12n oddn2n>0 evene(n) = \begin{cases} 0 & n = 0 \\ \frac{n+1}{2} & n \text{ odd} \\ -\frac{n}{2} & n > 0 \text{ even} \end{cases}

      This gives: e(0)=0e(0) = 0, e(1)=1e(1) = 1, e(2)=1e(2) = -1, e(3)=2e(3) = 2, e(4)=2e(4) = -2, \ldots

      Check: This is a bijection, hence surjection.

      Enumerable Implies Countable

      Every enumerable set is countable by definition.

      Non-empty countable sets are enumerable (finite = eventually constant enumeration).


      Summary

      • Enumerable: surjection NA\mathbb{N} \twoheadrightarrow A exists
      • Countable: finite or enumerable
      • Countable sets closed under subsets, sums, products
      • Axiom of Choice: every surjection has a section
      • Z\mathbb{Z}, N×N\mathbb{N} \times \mathbb{N}, and Q\mathbb{Q} are countable
    • Injections and Countability

      Injections

      Definition

      A function f:ABf : A \to B is injective if:

      a1,a2A.(f(a1)=f(a2)a1=a2)\forall a_1, a_2 \in A. (f(a_1) = f(a_2) \Rightarrow a_1 = a_2)

      Equivalently: a1a2f(a1)f(a2)a_1 \neq a_2 \Rightarrow f(a_1) \neq f(a_2).

      Notation

      Write f:ABf : A \hookrightarrow B for an injection.

      Injectivity and Cardinality

      Intuition

      An injection ABA \hookrightarrow B shows AA is “at most as large as” BB.

      For finite sets: if there is an injection ABA \hookrightarrow B, then AB|A| \leq |B|.

      For Infinite Sets

      An injection NA\mathbb{N} \hookrightarrow A shows AA is “at least as large as” N\mathbb{N}.

      Definition

      A set AA is countable if there exists an injection ANA \hookrightarrow \mathbb{N}.

      Equivalence: If AA \neq \emptyset, this is equivalent to: there exists a surjection NA\mathbb{N} \twoheadrightarrow A.

      Closure Properties

      Composition of Injections

      If f:ABf : A \hookrightarrow B and g:BCg : B \hookrightarrow C, then gf:ACg \circ f : A \hookrightarrow C.

      Proof: If (gf)(a1)=(gf)(a2)(g \circ f)(a_1) = (g \circ f)(a_2), then g(f(a1))=g(f(a2))g(f(a_1)) = g(f(a_2)). Since gg injective, f(a1)=f(a2)f(a_1) = f(a_2). Since ff injective, a1=a2a_1 = a_2.

      Injections and Products

      If f:ABf : A \hookrightarrow B and g:CDg : C \hookrightarrow D, then f×g:A×CB×Df \times g : A \times C \hookrightarrow B \times D.

      Cantor-Bernstein-Schroeder Theorem

      Statement

      If there exist injections f:ABf : A \hookrightarrow B and g:BAg : B \hookrightarrow A, then ABA \cong B.

      Significance

      This allows us to prove two sets have the same cardinality by constructing injections in both directions, without explicitly building a bijection.

      Proof Idea

      Using ff and gg, partition A=AAABA = A_A \cup A_B and B=BABBB = B_A \cup B_B such that:

      • ff bijects AABAA_A \cong B_A
      • g1g^{-1} bijects BBABB_B \cong A_B

      The union gives ABA \cong B.

      Well-Ordering and Choice

      Every Surjection Has a Section

      Given f:ABf : A \twoheadrightarrow B, for each bBb \in B, the fibre f1(b)Af^{-1}(b) \subseteq A is non-empty.

      A section chooses one element from each fibre.

      Axiom of Choice: Such a section exists for every surjection.

      Injectivity from Choice

      Given a section s:BAs : B \to A with fs=idBf \circ s = \text{id}_B, the section ss is injective.

      Proof: If s(b1)=s(b2)s(b_1) = s(b_2), then b1=f(s(b1))=f(s(b2))=b2b_1 = f(s(b_1)) = f(s(b_2)) = b_2.

      Enumeration via Injection

      Theorem

      For non-empty AA: AA is enumerable iff there exists an injection ANA \hookrightarrow \mathbb{N}.

      Proof Sketch:

      ()(\Rightarrow): If e:NAe : \mathbb{N} \twoheadrightarrow A is surjective, define s(a)=min{n:e(n)=a}s(a) = \min\{n : e(n) = a\}. This is injective (the section).

      ()(\Leftarrow): If i:ANi : A \hookrightarrow \mathbb{N} is injective, enumerate AA in order of increasing i(a)i(a).

      Countability of Q\mathbb{Q}

      Argument

      An injection QN×N\mathbb{Q} \hookrightarrow \mathbb{N} \times \mathbb{N}: send pq\frac{p}{q} (in lowest terms) to (p,q)(p, q).

      An injection N×NN\mathbb{N} \times \mathbb{N} \hookrightarrow \mathbb{N}: Cantor pairing function.

      Thus Q\mathbb{Q} is countable.

      Explicit Enumeration

      Enumration of positive rationals via Stern-Brocot tree or diagonal enumeration.


      Summary

      • Injection: distinct inputs give distinct outputs
      • AA countable iff injection ANA \hookrightarrow \mathbb{N} exists
      • Cantor-Bernstein-Schroeder: injections both ways implies bijection
      • Every surjection has a section (Axiom of Choice)
      • Q\mathbb{Q} is countable via embedding into N×N\mathbb{N} \times \mathbb{N}
    • The Cantor-Bernstein-Schroeder Theorem

      Statement

      Theorem (Cantor-Bernstein-Schroeder): If there exist injections f:ABf : A \hookrightarrow B and g:BAg : B \hookrightarrow A, then there exists a bijection h:ABh : A \cong B.

      Significance

      To prove ABA \cong B, we need only construct injections in both directions. The theorem guarantees a bijection exists implicitly.

      Equivalent Formulation

      If AB|A| \leq |B| and BA|B| \leq |A|, then A=B|A| = |B|.

      Proof Idea: The Ping-Pong Lemma

      Setup

      We have f:ABf : A \to B injective and g:BAg : B \to A injective.

      Goal: Partition A=A0A1A = A_0 \cup A_1 such that ff bijects A0A_0 with some B0B_0 and g1g^{-1} bijects A1A_1 with BB0B \setminus B_0.

      The Descending Chain Construction

      Define a sequence of sets:

      A0=Ag(B)A_0 = A \setminus g(B) An+1=g(f(An))A_{n+1} = g(f(A_n))

      And let A=n0AnA_{\infty} = \bigcup_{n \geq 0} A_n.

      Key Observation

      • ff is a bijection from AA_{\infty} to f(A)f(A_{\infty})
      • g1g^{-1} is a bijection from AAA \setminus A_{\infty} to Bf(A)B \setminus f(A_{\infty})

      The Bijection

      Define h:ABh : A \to B by:

      h(a)={f(a)aAg1(a)aAh(a) = \begin{cases} f(a) & a \in A_{\infty} \\ g^{-1}(a) & a \notin A_{\infty} \end{cases}

      Well-defined: If aAa \notin A_{\infty}, then ag(B)a \in g(B), so g1(a)g^{-1}(a) exists.

      Bijective: The two pieces are disjoint and hh is bijective on each.

      Detailed Proof

      Partition of AA

      Let:

      • D0=AIm(g)D_0 = A \setminus \text{Im}(g) (elements of AA not in the image of gg)
      • Dn+1=g(f(Dn))D_{n+1} = g(f(D_n)) (apply ff, then gg)
      • D=n0DnD = \bigcup_{n \geq 0} D_n

      Define the Bijection

      h(a)={f(a)aDg1(a)aDh(a) = \begin{cases} f(a) & a \in D \\ g^{-1}(a) & a \notin D \end{cases}

      Injectivity

      If h(a1)=h(a2)h(a_1) = h(a_2):

      • If both in DD: f(a1)=f(a2)f(a_1) = f(a_2), so a1=a2a_1 = a_2 (since ff injective).
      • If both in ADA \setminus D: g1(a1)=g1(a2)g^{-1}(a_1) = g^{-1}(a_2), so a1=a2a_1 = a_2 (since gg injective).
      • If one in DD, one not: Let a1Da_1 \in D, a2Da_2 \notin D. Then f(a1)=g1(a2)f(a_1) = g^{-1}(a_2), so g(f(a1))=a2g(f(D))=DD0g(f(a_1)) = a_2 \in g(f(D)) = D \setminus D_0. But a2Da_2 \notin D. Contradiction.

      Surjectivity

      Let bBb \in B.

      • If bf(D)b \in f(D): b=f(a)b = f(a) for some aDa \in D, so h(a)=bh(a) = b.
      • If bf(D)b \notin f(D): Then g(b)Dg(b) \notin D (otherwise g(b)Dng(b) \in D_n for some n>0n > 0, so bf(Dn1)f(D)b \in f(D_{n-1}) \subseteq f(D)). Thus h(g(b))=g1(g(b))=bh(g(b)) = g^{-1}(g(b)) = b.

      Alternative Proof Using Tarski’s Fixed Point Theorem

      The theorem follows from the Knaster-Tarski fixed point theorem on the complete lattice of subsets of AA.

      The operator Φ(S)=Ag(Bf(S))\Phi(S) = A \setminus g(B \setminus f(S)) has a fixed point SS^*.

      Then h=fSg1ASh = f|_{S^*} \cup g^{-1}|_{A \setminus S^*} is the desired bijection.

      Applications

      N×NN\mathbb{N} \times \mathbb{N} \cong \mathbb{N}

      • Injection NN×N\mathbb{N} \hookrightarrow \mathbb{N} \times \mathbb{N}: n(n,0)n \mapsto (n, 0)
      • Injection N×NN\mathbb{N} \times \mathbb{N} \hookrightarrow \mathbb{N}: Cantor pairing (a,b)(a+b)(a+b+1)2+b(a, b) \mapsto \frac{(a+b)(a+b+1)}{2} + b

      By CBS, N×NN\mathbb{N} \times \mathbb{N} \cong \mathbb{N}.

      RP(N)\mathbb{R} \cong \mathcal{P}(\mathbb{N})

      • Injection RP(N)\mathbb{R} \hookrightarrow \mathcal{P}(\mathbb{N}): send real to its binary expansion
      • Injection P(N)R\mathcal{P}(\mathbb{N}) \hookrightarrow \mathbb{R}: interpret subset as ternary expansion (0s and 2s only)

      By CBS, RP(N)\mathbb{R} \cong \mathcal{P}(\mathbb{N}).


      Summary

      • CBS: injections ABA \hookrightarrow B and BAB \hookrightarrow A imply ABA \cong B
      • Proof partitions AA and uses ff on one part, g1g^{-1} on the other
      • Allows proving cardinality equality without explicit bijection
    • Images and Image Factorisation

      Image of a Function

      Definition

      For f:ABf : A \to B, the image (or range) is:

      Im(f)={bB:aA.f(a)=b}={f(a):aA}\text{Im}(f) = \{b \in B : \exists a \in A. f(a) = b\} = \{f(a) : a \in A\}

      Image as a Surjection

      Define f~:AIm(f)\tilde{f} : A \twoheadrightarrow \text{Im}(f) by f~(a)=f(a)\tilde{f}(a) = f(a).

      This is surjective: for every bIm(f)b \in \text{Im}(f), there exists aa with f(a)=bf(a) = b.

      Image Factorisation

      Theorem

      Every function f:ABf : A \to B factors uniquely as:

      f=mff~f = m_f \circ \tilde{f}

      where:

      • f~:AIm(f)\tilde{f} : A \twoheadrightarrow \text{Im}(f) is surjective
      • mf:Im(f)Bm_f : \text{Im}(f) \hookrightarrow B is injective (the inclusion)

      Image factorisation of a function

      Verification

      For aAa \in A: mf(f~(a))=mf(f(a))=f(a)m_f(\tilde{f}(a)) = m_f(f(a)) = f(a) (since mfm_f is inclusion).

      Uniqueness

      If f=msf = m \circ s with ss surjective and mm injective, then Im(s)=Im(f)\text{Im}(s) = \text{Im}(f) and m=mfm = m_f up to isomorphism.

      Relational Images

      Direct Image

      For a relation R:ABR : A \nrightarrow B and XAX \subseteq A:

      R(X)={bB:xX.xRb}R_*(X) = \{b \in B : \exists x \in X. xRb\}

      This is the direct image of XX under RR.

      Inverse Image

      For YBY \subseteq B:

      R(Y)={aA:b.(aRbbY)}R^*(Y) = \{a \in A : \forall b. (aRb \Rightarrow b \in Y)\}

      This is the inverse image (or preimage) under RR.

      For Functions

      For f:ABf : A \to B:

      • f(X)={f(x):xX}f_*(X) = \{f(x) : x \in X\} (forward image)
      • f(Y)=f1(Y)={a:f(a)Y}f^*(Y) = f^{-1}(Y) = \{a : f(a) \in Y\} (inverse image)

      Properties of Inverse Images

      Preservation of Set Operations

      For f:ABf : A \to B:

      f1(Y1Y2)=f1(Y1)f1(Y2)f^{-1}(Y_1 \cap Y_2) = f^{-1}(Y_1) \cap f^{-1}(Y_2) f1(Y1Y2)=f1(Y1)f1(Y2)f^{-1}(Y_1 \cup Y_2) = f^{-1}(Y_1) \cup f^{-1}(Y_2) f1(Yc)=(f1(Y))cf^{-1}(Y^c) = (f^{-1}(Y))^c f1()=f^{-1}(\emptyset) = \emptyset f1(B)=Af^{-1}(B) = A

      Galois Connection

      There is a Galois connection between ff_* and ff^*:

      f(X)Y    Xf(Y)f_*(X) \subseteq Y \iff X \subseteq f^*(Y)

      Applications

      Surjectivity Characterisation

      ff is surjective iff Im(f)=B\text{Im}(f) = B iff f(A)=Bf_*(A) = B.

      Injectivity Characterisation

      ff is injective iff f(f(X))=Xf^*(f_*(X)) = X for all XAX \subseteq A.

      Bijection Characterisation

      ff is bijective iff ff_* and ff^* are mutually inverse bijections between P(A)\mathcal{P}(A) and P(B)\mathcal{P}(B).


      Summary

      • Im(f)={f(a):aA}\text{Im}(f) = \{f(a) : a \in A\}
      • Every f:ABf : A \to B factors as AIm(f)BA \twoheadrightarrow \text{Im}(f) \hookrightarrow B
      • Direct image f(X)={f(x):xX}f_*(X) = \{f(x) : x \in X\}
      • Inverse image f(Y)=f1(Y)={a:f(a)Y}f^*(Y) = f^{-1}(Y) = \{a : f(a) \in Y\}
      • Inverse images preserve all Boolean operations
    • Diagonalisation and Cantor's Theorem

      Cantor’s Theorem

      Statement

      Theorem (Cantor, 1891): For any set AA, there is no surjection AP(A)A \twoheadrightarrow \mathcal{P}(A).

      Corollary

      P(A)>A|\mathcal{P}(A)| > |A| for every set AA.

      In particular, there are different sizes of infinity.

      Examinable Proof (Diagonalisation)

      Setup

      Suppose e:AP(A)e : A \twoheadrightarrow \mathcal{P}(A) is surjective.

      The Diagonal Set

      Define UAU \subseteq A by:

      U={xA:xe(x)}U = \{x \in A : x \notin e(x)\}

      This is the set of all elements that are not members of the set they’re mapped to.

      Contradiction

      Since ee is surjective, there exists aAa \in A with e(a)=Ue(a) = U.

      Question: Is aUa \in U?

      • If aUa \in U: Then ae(a)=Ua \notin e(a) = U. Contradiction.
      • If aUa \notin U: Then ae(a)=Ua \in e(a) = U (by definition of UU). Contradiction.

      Either way, contradiction. Therefore no such surjection exists.

      Visualisation

      Consider A=NA = \mathbb{N}. If e:NP(N)e : \mathbb{N} \to \mathcal{P}(\mathbb{N}) enumerated all subsets:

      nne(n)Ne(n) \subseteq \mathbb{N}ne(n)n \in e(n)?
      0{1,2,3,}\{1, 2, 3, \ldots\}No
      1{0,2,4,}\{0, 2, 4, \ldots\}No
      2{0,1,3,}\{0, 1, 3, \ldots\}Yes
      3{3}\{3\}Yes
      \vdots\vdots\vdots

      The diagonal UU “flips” the membership on the diagonal. UU differs from every row, so cannot appear.

      Cantor diagonalisation visualisation

      Immediate Consequences

      Uncountability of R\mathbb{R}

      RP(N)\mathbb{R} \cong \mathcal{P}(\mathbb{N}) via binary expansions.

      Since P(N)\mathcal{P}(\mathbb{N}) is uncountable, R\mathbb{R} is uncountable.

      No Largest Cardinality

      Given any set AA, the set P(A)\mathcal{P}(A) is strictly larger.

      Iterating: A<P(A)<P(P(A))<|A| < |\mathcal{P}(A)| < |\mathcal{P}(\mathcal{P}(A))| < \ldots

      NR\mathbb{N} \subsetneq \mathbb{R}

      N\mathbb{N} is countable, R\mathbb{R} is not.

      Lawvere’s Fixed Point Theorem

      Statement

      If there exists a surjection e:A(AX)e : A \twoheadrightarrow (A \to X), then every function f:XXf : X \to X has a fixed point.

      Proof

      Define φ:AX\varphi : A \to X by φ(a)=f(e(a)(a))\varphi(a) = f(e(a)(a)).

      Since ee is surjective, there exists a0a_0 with e(a0)=φe(a_0) = \varphi.

      Then: e(a0)(a0)=φ(a0)=f(e(a0)(a0))e(a_0)(a_0) = \varphi(a_0) = f(e(a_0)(a_0)).

      So e(a0)(a0)e(a_0)(a_0) is a fixed point of ff.

      Deriving Cantor’s Theorem

      Take X={0,1}X = \{0, 1\}.

      A surjection A(A{0,1})A \twoheadrightarrow (A \to \{0, 1\}) would imply every f:{0,1}{0,1}f : \{0, 1\} \to \{0, 1\} has a fixed point.

      But the swap map f(0)=1f(0) = 1, f(1)=0f(1) = 0 has no fixed point. Contradiction.

      Therefore (A{0,1})P(A)(A \to \{0, 1\}) \cong \mathcal{P}(A) has no surjection from AA.


      Summary

      • Cantor’s Theorem: No surjection AP(A)A \twoheadrightarrow \mathcal{P}(A)
      • Proof uses diagonal argument: construct U={x:xe(x)}U = \{x : x \notin e(x)\}
      • R\mathbb{R} is uncountable; there is no largest set
      • Lawvere’s theorem generalises Cantor: surjections to function spaces force fixed points
    • Well-Foundedness

      Minimal Elements

      Definition

      Let \prec be a relation on AA. An element mSAm \in S \subseteq A is minimal in SS if:

      ¬xS.xm\neg\exists x \in S. x \prec m

      No element of SS is strictly below mm.

      Distinction: Minimal vs Least

      • Minimal: No element is below mm (local property)
      • Least: mm is below every element of SS

      A set can have multiple minimal elements but at most one least element.

      Example

      On {1,2,3}\{1, 2, 3\} with partial order given by divisibility:

      • Minimal elements of {2,3,4,6}\{2, 3, 4, 6\} are 22 and 33
      • No least element

      Well-Founded Relations

      Definition

      A relation \prec on AA is well-founded if every non-empty subset SAS \subseteq A has a minimal element.

      SA.(SmS.¬xS.xm)\forall S \subseteq A. (S \neq \emptyset \Rightarrow \exists m \in S. \neg\exists x \in S. x \prec m)

      Equivalent Characterisation

      \prec is well-founded iff there is no infinite descending chain:

      a0a1a2a_0 \succ a_1 \succ a_2 \succ \cdots

      Proof:

      ()(\Rightarrow): If such a chain exists, the set {a0,a1,a2,}\{a_0, a_1, a_2, \ldots\} has no minimal element.

      ()(\Leftarrow): If some SS has no minimal element, we can build a descending chain by repeatedly choosing smaller elements.

      Examples

      Well-Founded

      • << on N\mathbb{N}: Every non-empty subset has a least element (well-order)
      • << on Z\mathbb{Z} restricted to {n:nk}\{n : n \geq k\} for any kk
      • The “proper subterm” relation on finite trees

      Not Well-Founded

      • << on Z\mathbb{Z}: {,2,1,0}\{\ldots, -2, -1, 0\} has no minimal element
      • << on Q(0,1)\mathbb{Q} \cap (0, 1): {1/n:n1}\{1/n : n \geq 1\} has no minimal element
      • << on R\mathbb{R}: any unbounded-below subset

      Well-Orders

      Definition

      A well-order is a well-founded total order.

      Every non-empty subset has a unique least element.

      Examples

      • (N,)(\mathbb{N}, \leq) is a well-order
      • (Z,)(\mathbb{Z}, \leq) is NOT a well-order
      • Any finite totally ordered set is a well-order

      Properties

      If \prec is a well-order, then:

      1. Every non-empty subset has a unique least element
      2. For each aa, the set {x:xa}\{x : x \prec a\} is well-ordered
      3. Principle of induction applies (see next section)

      The Axiom of Foundation

      In ZF set theory, the Axiom of Foundation states that \in is well-founded on any set.

      This prohibits infinite descending \in-chains: x2x1x0\cdots \in x_2 \in x_1 \in x_0


      Summary

      • mm is minimal in SS if no element of SS is below mm
      • \prec is well-founded if every non-empty subset has a minimal element
      • Equivalent: no infinite descending chains
      • << on N\mathbb{N} is well-founded; << on Z\mathbb{Z} or Q\mathbb{Q} is not
      • A well-order is a well-founded total order
    • Well-Founded Induction

      The Principle of Well-Founded Induction

      Statement

      Let \prec be a well-founded relation on AA. Let PP be a property. If:

      xA.(yx.P(y))P(x)\forall x \in A. (\forall y \prec x. P(y)) \Rightarrow P(x)

      then xA.P(x)\forall x \in A. P(x) holds.

      Intuition

      To prove P(x)P(x) for all xx, we may assume P(y)P(y) holds for all yxy \prec x when proving P(x)P(x).

      This generalises strong induction from N\mathbb{N} to any well-founded structure.

      Examinable Proof

      Theorem: Well-founded induction is valid.

      Proof:

      Suppose xA.(yx.P(y))P(x)\forall x \in A. (\forall y \prec x. P(y)) \Rightarrow P(x) holds.

      Assume for contradiction that S={xA:¬P(x)}S = \{x \in A : \neg P(x)\} is non-empty.

      Since \prec is well-founded, SS has a minimal element mm.

      Key observation: mSm \in S implies ¬P(m)\neg P(m), but ym.P(y)\forall y \prec m. P(y) holds (since mm is minimal in SS).

      By assumption, (ym.P(y))P(m)(\forall y \prec m. P(y)) \Rightarrow P(m).

      This gives P(m)P(m), contradicting mSm \in S.

      Therefore S=S = \emptyset, i.e., x.P(x)\forall x. P(x).

      Connection to Strong Induction

      On N\mathbb{N}

      Strong induction on N\mathbb{N} is exactly well-founded induction for   =  <\prec \; = \; < on N\mathbb{N}.

      The premise becomes: n.(m<n.P(m))P(n)\forall n. (\forall m < n. P(m)) \Rightarrow P(n).

      Base Case

      For n=0n = 0: the hypothesis m<0.P(m)\forall m < 0. P(m) is vacuously true, so we must prove P(0)P(0) directly.

      This recovers the base case of strong induction.

      Using Well-Founded Induction

      Recipe

      1. Identify the well-founded relation \prec
      2. State the induction hypothesis: yx.P(y)\forall y \prec x. P(y)
      3. Prove P(x)P(x) assuming the hypothesis

      Example: Termination of Euclid’s Algorithm

      Claim: gcd(a,b)\gcd(a, b) terminates for all a,bNa, b \in \mathbb{N} (not both zero).

      Well-founded relation: Lexicographic ordering on pairs (a,b)(a, b).

      Proof: At each step, (a,b)(b,amodb)(a, b) \mapsto (b, a \bmod b).

      Since amodb<ba \bmod b < b, the second component decreases (or stays zero while the first decreases).

      By well-foundedness of the lexicographic order, the process terminates.

      Lexicographic Ordering

      Definition

      For well-founded relations A\prec_A on AA and B\prec_B on BB, define the lexicographic order on A×BA \times B:

      (a1,b1)(a2,b2)    a1Aa2(a1=a2b1Bb2)(a_1, b_1) \prec (a_2, b_2) \iff a_1 \prec_A a_2 \lor (a_1 = a_2 \land b_1 \prec_B b_2)

      Well-Foundedness

      If A\prec_A and B\prec_B are well-founded, then \prec (lexicographic) is well-founded.

      Application

      This justifies nested induction or induction on pairs.

      Induction on Recursive Structures

      Trees

      For finite trees with subtree relation “is proper subtree of”:

      This is well-founded (trees have finite depth).

      Lists

      For lists with “tail” operation:

      “Is a proper suffix of” is well-founded.


      Summary

      • Well-founded induction: if (yx.P(y))P(x)(\forall y \prec x. P(y)) \Rightarrow P(x) for all xx, then x.P(x)\forall x. P(x)
      • Proof: minimal element of counterexample set leads to contradiction
      • Specialises to strong induction on N\mathbb{N}
      • Lexicographic order preserves well-foundedness
  • Formal Languages

    Alphabets, strings, inductive definitions, parse trees, rule induction, and regular expressions

    • Alphabets and Strings

      Alphabets

      Definition

      An alphabet is a finite set Σ\Sigma whose elements are called symbols.

      Examples

      • {0,1}\{0, 1\}: binary alphabet
      • {a,b,c,,z}\{a, b, c, \ldots, z\}: lowercase English letters
      • {0,1,2,,9}\{0, 1, 2, \ldots, 9\}: decimal digits
      • ASCII character set (finite)
      • {}=\{\} = \emptyset: the empty alphabet

      Non-examples

      • N\mathbb{N}: infinite, not an alphabet
      • R\mathbb{R}: infinite, not an alphabet

      Strings

      Definition

      A string (or word) over alphabet Σ\Sigma is a finite sequence of symbols from Σ\Sigma.

      • Strings are written without punctuation: abbaabba, 00110011
      • The length of string ww is written w|w|
      • The empty string (length 0) is written ε\varepsilon or λ\lambda

      Formal Definition

      A string of length nn is a function w:[n]Σw : [n] \to \Sigma, where [n]={0,1,,n1}[n] = \{0, 1, \ldots, n-1\}.

      The empty string is the unique function [0]Σ[0] \to \Sigma.

      Examples

      Over Σ={a,b}\Sigma = \{a, b\}:

      • ε\varepsilon: empty string, ε=0|\varepsilon| = 0
      • aa: length 1
      • abbaabba: length 4
      • aaaaaaaa: length 4

      The Set of All Strings

      Definition

      Σ\Sigma^* is the set of all strings over Σ\Sigma, including ε\varepsilon.

      Σ={w:w is a string over Σ}=n0Σn\Sigma^* = \{w : w \text{ is a string over } \Sigma\} = \bigcup_{n \geq 0} \Sigma^n

      where Σn={w:w=n}\Sigma^n = \{w : |w| = n\} is the set of length-nn strings.

      Special Cases

      • If Σ=\Sigma = \emptyset: Σ={ε}\Sigma^* = \{\varepsilon\} (only the empty string)
      • If Σ={a}\Sigma = \{a\}: Σ={ε,a,aa,aaa,}\Sigma^* = \{\varepsilon, a, aa, aaa, \ldots\}

      Distinguishing Notation

      SymbolMeaning
      \emptysetThe empty set
      {ε}\{\varepsilon\}The set containing the empty string
      ε\varepsilonThe empty string itself

      These are three distinct objects!

      Concatenation

      Definition

      The concatenation of strings uu and vv, written uvuv, is the string formed by appending vv to uu.

      uv=u+v|uv| = |u| + |v|

      Properties

      • Identity: uε=u=εuu\varepsilon = u = \varepsilon u
      • Associativity: (uv)w=u(vw)(uv)w = u(vw)

      Concatenation is NOT commutative: in general uvvuuv \neq vu.

      Examples

      If u=abu = ab and v=bav = ba:

      • uv=abbauv = abba
      • vu=baabvu = baab
      • uu=ababuu = abab
      • u3=abababu^3 = ababab

      Powers

      For a string uu and n0n \geq 0:

      un=uuun timesu^n = \underbrace{uu\cdots u}_{n \text{ times}}

      By convention, u0=εu^0 = \varepsilon.

      Strings Form a Monoid

      Algebraic Structure

      (Σ,,ε)(\Sigma^*, \cdot, \varepsilon) is a monoid:

      • Binary operation: concatenation
      • Identity element: ε\varepsilon
      • Associative: (uv)w=u(vw)(uv)w = u(vw)

      Not a Group

      Strings do not generally have inverses under concatenation.

      If uv=εuv = \varepsilon, then u=v=εu = v = \varepsilon.

      Free Monoid

      Σ\Sigma^* is the free monoid on Σ\Sigma: every monoid generated by Σ\Sigma is a quotient of Σ\Sigma^*.

      Substrings and Prefixes

      Substring

      uu is a substring of ww if w=xuyw = xuy for some strings x,yx, y.

      Prefix

      uu is a prefix of ww if w=uvw = uv for some string vv.

      Suffix

      uu is a suffix of ww if w=vuw = vu for some string vv.

      Examples

      For w=abcdw = abcd:

      • Prefixes: ε,a,ab,abc,abcd\varepsilon, a, ab, abc, abcd
      • Suffixes: ε,d,cd,bcd,abcd\varepsilon, d, cd, bcd, abcd
      • Substrings include: bcbc, bb, ε\varepsilon, abcdabcd

      Summary

      • An alphabet Σ\Sigma is a finite set of symbols
      • Σ\Sigma^* is the set of all finite strings over Σ\Sigma
      • Concatenation is associative with identity ε\varepsilon
      • (Σ,,ε)(\Sigma^*, \cdot, \varepsilon) forms a monoid
      • \emptyset, {ε}\{\varepsilon\}, and ε\varepsilon are distinct
    • Formal Languages

      Definition

      A formal language (or just language) over alphabet Σ\Sigma is any subset LΣL \subseteq \Sigma^*.

      Intuition

      A language is specified by giving the “words in the dictionary” - the strings that belong to it.

      Examples

      LanguageAlphabetDescription
      \emptysetanyEmpty language (no strings)
      {ε}\{\varepsilon\}anyLanguage containing only the empty string
      Σ\Sigma^*Σ\SigmaAll strings over Σ\Sigma
      {anbn:n0}\{a^n b^n : n \geq 0\}{a,b}\{a, b\}Equal numbers of aa‘s and bb‘s
      ${w :w\text{ even}}$

      Operations on Languages

      Union

      L1L2={w:wL1wL2}L_1 \cup L_2 = \{w : w \in L_1 \lor w \in L_2\}

      Intersection

      L1L2={w:wL1wL2}L_1 \cap L_2 = \{w : w \in L_1 \land w \in L_2\}

      Complement

      Lc=ΣL={wΣ:wL}L^c = \Sigma^* \setminus L = \{w \in \Sigma^* : w \notin L\}

      Concatenation of Languages

      Definition

      L1L2={uv:uL1vL2}L_1 \cdot L_2 = \{uv : u \in L_1 \land v \in L_2\}

      Properties

      • Identity: L{ε}=L={ε}LL \cdot \{\varepsilon\} = L = \{\varepsilon\} \cdot L
      • Associativity: (L1L2)L3=L1(L2L3)(L_1 \cdot L_2) \cdot L_3 = L_1 \cdot (L_2 \cdot L_3)
      • Distributive over union: L1(L2L3)=(L1L2)(L1L3)L_1 \cdot (L_2 \cup L_3) = (L_1 \cdot L_2) \cup (L_1 \cdot L_3)

      Special Cases

      • L==LL \cdot \emptyset = \emptyset = \emptyset \cdot L
      • {ε}L=L\{\varepsilon\} \cdot L = L

      Kleene Star

      Definition

      The Kleene star of language LL is:

      L=n0Ln={ε}LL2L3L^* = \bigcup_{n \geq 0} L^n = \{\varepsilon\} \cup L \cup L^2 \cup L^3 \cup \cdots

      where Ln=LLLn timesL^n = \underbrace{L \cdot L \cdot \ldots \cdot L}_{n \text{ times}}.

      Alternative Definition

      LL^* is the smallest language containing {ε}\{\varepsilon\} and LL, closed under concatenation.

      Examples

      • {a}={ε,a,aa,aaa,}\{a\}^* = \{\varepsilon, a, aa, aaa, \ldots\}
      • {a,b}=Σ\{a, b\}^* = \Sigma^* if Σ={a,b}\Sigma = \{a, b\}
      • ={ε}\emptyset^* = \{\varepsilon\} (only the empty string)
      • {ε}={ε}\{\varepsilon\}^* = \{\varepsilon\}

      Kleene Plus

      L+=LL=L{ε}=n1LnL^+ = L \cdot L^* = L^* \setminus \{\varepsilon\} = \bigcup_{n \geq 1} L^n

      Language Properties

      Finite vs Infinite Languages

      A language is finite if L<|L| < \infty, otherwise infinite.

      Examples:

      • {a,aa,aaa}\{a, aa, aaa\}: finite (3 elements)
      • {an:n0}\{a^n : n \geq 0\}: infinite

      Decidable Languages

      A language LL is decidable if there is an algorithm that determines whether any wΣw \in \Sigma^* belongs to LL.

      Closure Properties

      Languages are closed under:

      OperationResult
      UnionL1L2L_1 \cup L_2
      IntersectionL1L2L_1 \cap L_2
      ComplementLcL^c
      ConcatenationL1L2L_1 \cdot L_2
      Kleene starLL^*

      Special Languages

      The Empty Language

      L=L = \emptyset: contains no strings.

      Note: {ε}\emptyset \neq \{\varepsilon\}.

      The Universal Language

      L=ΣL = \Sigma^*: contains all strings.

      Singleton Languages

      L={w}L = \{w\}: contains exactly one string.


      Summary

      • A formal language is any subset LΣL \subseteq \Sigma^*
      • Operations: union, intersection, complement, concatenation, Kleene star
      • L=n0LnL^* = \bigcup_{n \geq 0} L^n is the Kleene closure
      • Languages form a Boolean algebra under \cup, \cap, complement
    • Inductive Definitions

      Motivation

      Many important sets are described inductively: we specify:

      1. Axioms: certain elements are in the set by fiat
      2. Rules: ways to derive new elements from existing ones

      This is more flexible than explicit enumeration or comprehension.

      Axioms and Rules

      Axioms

      An axiom is a statement of the form: “element aa is in the set”.

      We write axioms as horizontal lines with nothing above:

      a\frac{}{a}

      Rules

      A rule states: if u1,,unu_1, \ldots, u_n are in the set, then vv is in the set.

      We write rules as:

      u1u2unv\frac{u_1 \quad u_2 \quad \cdots \quad u_n}{v}

      • The uiu_i are called premises (or hypotheses)
      • vv is called the conclusion
      • A rule with no premises is an axiom

      Closure Conditions

      Given a set of rules RR, each rule defines a closure condition:

      {SX:u1,,unSvS}\{S \subseteq X : u_1, \ldots, u_n \in S \Rightarrow v \in S\}

      A set SS satisfies all closure conditions iff it is “closed under the rules.”

      Defining Sets Inductively

      Method

      Given a set XX and a set of rules RR:

      1. Start with the axioms
      2. Repeatedly apply rules to derive new elements
      3. The inductively defined set XRX_R contains all derivable elements

      Formal Definition

      Let ClR\text{Cl}_R be the collection of all subsets of XX closed under RR.

      XR=SClRSX_R = \bigcap_{S \in \text{Cl}_R} S

      This is the smallest subset of XX closed under RR.

      Why Smallest?

      • XRX_R itself is closed under RR (intersection preserves closure)
      • Any SS closed under RR contains XRX_R (if not, some element of XRX_R would be missing from SS)

      Example: Natural Numbers

      Define N\mathbb{N} inductively with rules:

      0nS(n)\frac{}{0} \qquad \frac{n}{S(n)}

      Interpretation

      • 0N0 \in \mathbb{N} (axiom)
      • If nNn \in \mathbb{N}, then S(n)NS(n) \in \mathbb{N} (successor rule)
      • Nothing else is in N\mathbb{N} (smallest such set)

      Derived Elements

      0,S(0),S(S(0)),S(S(S(0))),0, S(0), S(S(0)), S(S(S(0))), \ldots are all in N\mathbb{N}.

      Example: Binary Trees

      Define trees with rules:

      LeafT1T2Node(T1,T2)\frac{}{\text{Leaf}} \qquad \frac{T_1 \quad T_2}{\text{Node}(T_1, T_2)}

      Derived Elements

      Leaf, Node(Leaf, Leaf), Node(Node(Leaf, Leaf), Leaf), …

      Example: A Language

      Define {anbn:n0}\{a^n b^n : n \geq 0\} with rules:

      εwawb\frac{}{\varepsilon} \qquad \frac{w}{awb}

      Derived Elements

      εL\varepsilon \in L, so aεb=abLa\varepsilon b = ab \in L, so aabbLaabb \in L, so aaabbbLaaabbb \in L, …

      Inductively Defined Functions

      Setup

      Given aAa \in A and f:N×AAf : \mathbb{N} \times A \to A, define ρa,f:NA\rho_{a,f} : \mathbb{N} \to A by:

      ρa,f(0)=a\rho_{a,f}(0) = a ρa,f(n+1)=f(n,ρa,f(n))\rho_{a,f}(n+1) = f(n, \rho_{a,f}(n))

      Existence and Uniqueness

      The function ρa,f\rho_{a,f} exists and is unique.

      Proof idea: Define ρa,f\rho_{a,f} as the intersection of all (a,f)(a, f)-closed relations, then show it is total and functional.


      Summary

      • Inductive definition: specify axioms and rules
      • The defined set XRX_R is the smallest set closed under all rules
      • XR={SX:S closed under R}X_R = \bigcap\{S \subseteq X : S \text{ closed under } R\}
      • Examples: N\mathbb{N}, trees, languages, recursive functions
    • Derivations and Derivation Trees

      Derivations

      Definition

      A derivation (or proof) that vXRv \in X_R is a finite tree where:

      1. The root is labelled vv
      2. Each node is the conclusion of a rule whose premises are its children
      3. Each leaf is an axiom

      Intuition

      A derivation shows how vv can be built by applying rules starting from axioms.

      Derivation Trees

      Structure

              v
             /|\
            / | \
          u1  u2  u3
          /|   |   \
         ...  ...  ...
      • Root at top (conclusion)
      • Leaves at bottom (axioms)
      • Internal nodes: applications of rules

      Depth and Height

      • The height of a derivation is the length of the longest path from root to leaf
      • The depth of a node is its distance from the root

      Example: Natural Numbers

      Derivation of S(S(0))S(S(0)):

          S(S(0))
             |
           S(0)
             |
            0

      Reading

      1. 0N0 \in \mathbb{N} (axiom)
      2. S(0)NS(0) \in \mathbb{N} (by successor rule from 0)
      3. S(S(0))NS(S(0)) \in \mathbb{N} (by successor rule from S(0)S(0))

      Example: Binary Trees

      Derivation of Node(Leaf,Node(Leaf,Leaf))\text{Node}(\text{Leaf}, \text{Node}(\text{Leaf}, \text{Leaf})):

              Node(Leaf, Node(Leaf, Leaf))
                    /              \
                Leaf          Node(Leaf, Leaf)
                                /        \
                             Leaf        Leaf

      The Set of Derivable Elements

      Definition

      XRXX_R \subseteq X is the set of elements that have a derivation using rules RR.

      Key Theorem

      XR={SX:S closed under R}X_R = \bigcap \{S \subseteq X : S \text{ closed under } R\}

      The set of derivable elements equals the smallest closed set.

      Types of Rules

      Axioms (0-premise rules)

      a\frac{}{a}

      Elements derived in one step.

      Unary Rules (1-premise)

      uv\frac{u}{v}

      If uu derivable, so is vv.

      Binary Rules (2-premises)

      u1u2v\frac{u_1 \quad u_2}{v}

      If both u1u_1 and u2u_2 derivable, so is vv.

      n-ary Rules

      u1u2unv\frac{u_1 \quad u_2 \quad \cdots \quad u_n}{v}

      Reasoning About Derivations

      By Induction on Derivation Height

      To prove vXR.P(v)\forall v \in X_R. P(v):

      1. Base: Prove P(a)P(a) for every axiom aa
      2. Step: For each rule u1,,unv\frac{u_1, \ldots, u_n}{v}, assume P(u1),,P(un)P(u_1), \ldots, P(u_n) and prove P(v)P(v)

      This is a powerful proof technique that mirrors the structure of derivations.

      Example: Balanced Strings

      Rules for {w{a,b}:wa=wb}\{w \in \{a, b\}^* : |w|_a = |w|_b\}:

      εwawbwbwauvuv\frac{}{\varepsilon} \qquad \frac{w}{awb} \qquad \frac{w}{bwa} \qquad \frac{u \quad v}{uv}

      Claim: All derivable strings have equal numbers of aa‘s and bb‘s.

      Proof by induction on derivations:

      • Base: ε\varepsilon has 00 aa‘s and 00 bb‘s.
      • If ww balances, so do awbawb and bwabwa.
      • If uu and vv balance, so does uvuv (sums add).

      Summary

      • A derivation is a finite tree with axioms at leaves and rule applications at internal nodes
      • XRX_R = set of elements with a derivation
      • Induction on derivation height: base case for axioms, step case for each rule
    • Rule Induction

      The Rule Induction Principle

      Statement

      Let RR be a set of rules on XX. Let XRX_R be the set of derivable elements.

      To prove xXR.P(x)\forall x \in X_R. P(x): show that for every rule

      u1unv\frac{u_1 \quad \cdots \quad u_n}{v}

      if P(ui)P(u_i) holds for all premises uiu_i, then P(v)P(v) holds.

      Intuition

      If every rule “preserves” property PP from premises to conclusion, and PP holds for all axioms (rules with no premises), then PP holds for all derivable elements.

      Examinable Proof: Rule Induction is Valid

      Theorem: If S={xX:P(x)}S = \{x \in X : P(x)\} is closed under all rules in RR, then XRSX_R \subseteq S.

      Proof:

      Method: Induction on the height of derivations.

      Base case (height 0\leq 0): No derivation has height 00 (a derivation must be a tree with at least a root). Alternatively: axioms have height 11, and SS contains all axiom conclusions since SS is closed under rules with no premises.

      Inductive step: Suppose all derivations of height n\leq n have conclusions in SS. Consider a derivation of height n+1n+1 with conclusion vv.

      The derivation uses some rule u1unv\frac{u_1 \quad \cdots \quad u_n}{v} where each uiu_i is derived by a subtree of height n\leq n.

      By the induction hypothesis, each uiSu_i \in S.

      Since SS is closed under the rule, vSv \in S.

      Conclusion: By induction, every derivable element is in SS.

      Equivalence of Characterisations

      Two Characterisations of XRX_R

      1. XR={x:x has a derivation using R}X_R = \{x : x \text{ has a derivation using } R\}
      2. XR={S:S closed under R}X_R = \bigcap \{S : S \text{ closed under } R\}

      Proof of Equivalence

      (12)(1 \subseteq 2): Every SS closed under RR contains all derivable elements (shown by induction on derivation height).

      (21)(2 \subseteq 1): XRX_R is itself closed under RR (union of rule applications preserves membership), and contains exactly the derivable elements.

      The Recipe for Rule Induction

      To prove xXR.P(x)\forall x \in X_R. P(x):

      1. For each axiom a\frac{}{a}: verify P(a)P(a) holds
      2. For each rule u1unv\frac{u_1 \quad \cdots \quad u_n}{v}: assume P(u1),,P(un)P(u_1), \ldots, P(u_n) and prove P(v)P(v)

      This is the single most reusable proof template in this course.

      Example: Balanced Parentheses

      Rules

      εuvu(v)\frac{}{\varepsilon} \qquad \frac{u \quad v}{u(v)}

      Define language LRL_R with these rules.

      Claim

      Every wLRw \in L_R has balanced parentheses: #(’(’) = #(’)’) and every prefix has at least as many ’(’ as ’)’.

      Proof by Rule Induction

      Axiom: ε\varepsilon has 0 ’(’ and 0 ’)’. Trivially balanced.

      Rule: Assume uu and vv are balanced. Consider u(v)u(v).

      • Count: adds 1 ’(’ and 1 ’)’ to total from uu and vv. Since u,vu, v balanced, total balanced.
      • Prefix property: Any prefix is either:
        • A prefix of uu (balanced by IH)
        • uu itself (balanced by IH)
        • u(u( or u(vu(v' where vv' is a prefix of vv (still satisfies prefix property)

      Example: Natural Numbers

      Rules

      0nS(n)\frac{}{0} \qquad \frac{n}{S(n)}

      Claim: Every derived nn is either 00 or S(m)S(m) for some derived mm

      Proof:

      • Axiom: 00 is of the right form.
      • Rule: S(n)S(n) is of the right form (as successor of nn).

      When to Use Rule Induction

      Use rule induction when:

      1. The set is defined inductively by rules
      2. The property PP should be preserved by rule application
      3. You need to prove something about all derivable elements

      Common applications:

      • Properties of syntax trees
      • Type preservation in programming languages
      • Semantic equivalences
      • Language properties

      Summary

      • Rule induction: if SS closed under RR, then XRSX_R \subseteq S
      • Recipe: handle axioms, then each rule preserves PP from premises to conclusion
      • Proof: induction on derivation height
      • Fundamental proof technique for inductively defined sets
    • Concrete and Abstract Syntax

      The Distinction

      Concrete Syntax

      Concrete syntax is how expressions are written as text strings by programmers. It includes:

      • Parentheses for grouping
      • Operator precedence
      • Associativity rules
      • Whitespace and formatting

      Abstract Syntax

      Abstract syntax is the “essential structure” of an expression, typically represented as a tree. It captures:

      • The operators and operands
      • The hierarchical structure
      • No need for parentheses or precedence

      Example: Arithmetic Expressions

      Concrete Syntax

      The string "1 + 2 * 3" has a specific meaning due to precedence (* binds tighter than +).

      Different strings can have the same meaning:

      • "1 + 2 * 3"
      • "1+(2*3)"
      • "1 + ( 2 * 3 )"

      Abstract Syntax Tree

      All three concrete strings map to the same abstract syntax tree:

      Abstract syntax tree for 1 + 2 * 3

          +
         / \
        1   *
           / \
          2   3

      Parsing

      Definition

      Parsing is the process of converting concrete syntax (a string) to abstract syntax (a tree).

      Grammar Rules

      Expressions can be defined by grammar rules:

      E ::= E + T | T
      T ::= T * F | F
      F ::= n | ( E )

      This grammar enforces precedence: E handles addition, T handles multiplication.

      Ambiguity

      A grammar is ambiguous if some string has multiple parse trees.

      This is usually undesirable: the meaning should be unique.

      Abstract Syntax Trees for Regular Expressions

      Constructors

      For regular expressions over alphabet Σ\Sigma:

      ConstructorNotationMeaning
      Nullε\varepsilonEmpty string
      Never\emptysetNo strings
      Symbolaa for aΣa \in \SigmaSingle symbol
      Union$R_1R_2$
      ConcatR1R2R_1 R_2Sequencing
      StarRR^*Kleene closure

      Tree Representation

      The regex a(b|c)* has AST:

          Concat
          /    \
         a     Star
                |
               Union
               /  \
              b    c

      Parsing Regular Expressions

      Precedence

      From highest to lowest:

      1. Star (*): binds tightest
      2. Concatenation (implicit): next
      3. Union (|): binds loosest

      Associativity

      • Star: postfix, right-associative
      • Concat: implicit, left-associative
      • Union: left-associative (usually)

      Example

      ab|c*d parses as (ab)|((c*)d):

              Union
             /    \
          Concat  Concat
          /   \   /   \
         a     b Star  d
                  |
                  c

      The Parsing Relation

      Definition

      Define a relation _ parsesTo _ between strings and ASTs.

      Rules

      Atoms: "ε" parsesTo Null"" parsesTo Nevera parsesTo Syma\frac{}{\text{"}\varepsilon\text{" parsesTo Null}} \qquad \frac{}{\text{"}\emptyset\text{" parsesTo Never}} \qquad \frac{}{a \text{ parsesTo Sym}_a}

      Compound: s1 parsesTo R1s2 parsesTo R2s1s2 parsesTo Union(R1,R2)\frac{s_1 \text{ parsesTo } R_1 \quad s_2 \text{ parsesTo } R_2}{s_1 | s_2 \text{ parsesTo Union}(R_1, R_2)}

      Similar for Concat and Star.

      Pretty Printing

      Definition

      Pretty printing is the inverse of parsing: converting an AST to a readable string.

      Challenge

      The string should be:

      • Unambiguous
      • Readable (minimal parentheses)

      Algorithm

      Add parentheses only when necessary based on precedence.

      If child’s precedence is lower than parent’s, parenthesize the child.


      Summary

      • Concrete syntax: the textual representation
      • Abstract syntax: the tree structure (AST)
      • Parsing: concrete to abstract
      • Pretty printing: abstract to concrete
      • Precedence resolves ambiguity in concrete syntax
    • Regular Expressions

      Definition

      Abstract Syntax

      Regular expressions over alphabet Σ\Sigma are defined inductively:

      ConstructorSyntaxMeaning
      Nullε\varepsilonMatches the empty string only
      Never\emptysetMatches nothing
      Symbolaa for aΣa \in \SigmaMatches the single symbol aa
      Union$R_1R_2$
      ConcatR1R2R_1 \cdot R_2 or R1R2R_1 R_2Matches R1R_1 followed by R2R_2
      StarRR^*Matches zero or more repetitions of RR

      Concrete Syntax

      The typical concrete syntax uses:

      • | for union (alternation)
      • Juxtaposition for concatenation: ab means aa then bb
      • * for Kleene star
      • () for grouping

      Precedence (highest to lowest)

      1. Star (*)
      2. Concatenation (implicit)
      3. Union (|)

      The Language of a Regular Expression

      Definition

      Each regex RR denotes a language L(R)ΣL(R) \subseteq \Sigma^*, defined inductively:

      Regex RRLanguage L(R)L(R)
      ε\varepsilon{ε}\{\varepsilon\}
      \emptyset\emptyset
      aa{a}\{a\}
      $R_1R_2$
      R1R2R_1 R_2L(R1)L(R2)={uv:uL(R1),vL(R2)}L(R_1) \cdot L(R_2) = \{uv : u \in L(R_1), v \in L(R_2)\}
      RR^*L(R)=n0L(R)nL(R)^* = \bigcup_{n \geq 0} L(R)^n

      Examples

      Single Symbols

      • L(a)={a}L(a) = \{a\}
      • L(ab)={ab}L(ab) = \{ab\}
      • L(ab)={a,b}L(a|b) = \{a, b\}

      Kleene Star

      • L(a)={ε,a,aa,aaa,}L(a^*) = \{\varepsilon, a, aa, aaa, \ldots\}
      • L((ab))=ΣL((a|b)^*) = \Sigma^* (all strings over {a,b}\{a, b\})
      • L()={ε}L(\emptyset^*) = \{\varepsilon\}

      Combinations

      • L(ab)={anbm:n,m0}L(a^*b^*) = \{a^n b^m : n, m \geq 0\}
      • L((ab))={ε,ab,abab,ababab,}L((ab)^*) = \{\varepsilon, ab, abab, ababab, \ldots\}
      • L(a(bc))={a,ab,ac,abb,abc,acb,acc,}L(a(b|c)^*) = \{a, ab, ac, abb, abc, acb, acc, \ldots\}

      Idioms and Patterns

      Every String

      (ab)=Σ (when Σ={a,b})(a|b)^* = \Sigma^* \text{ (when } \Sigma = \{a, b\})

      Strings Starting with aa

      a(ab)a(a|b)^*

      Strings Ending with aa

      (ab)a(a|b)^*a

      Strings Containing aa

      (ab)a(ab)(a|b)^*a(a|b)^*

      Even-Length Strings

      ((ab)(ab))((a|b)(a|b))^*

      At Least One aa

      (ab)a(ab)(a|b)^*a(a|b)^*

      Exactly One aa

      babb^*ab^*

      Algebraic Laws

      Union

      • Commutative: RS=SRR|S = S|R
      • Associative: (RS)T=R(ST)(R|S)|T = R|(S|T)
      • Identity: R=RR|\emptyset = R
      • Idempotent: RR=RR|R = R

      Concatenation

      • Associative: (RS)T=R(ST)(RS)T = R(ST)
      • Identity: Rε=R=εRR\varepsilon = R = \varepsilon R
      • Zero: R==RR\emptyset = \emptyset = \emptyset R
      • Distributive over union: R(ST)=RSRTR(S|T) = RS|RT and (RS)T=RTST(R|S)T = RT|ST

      Star

      • ε=ε\varepsilon^* = \varepsilon
      • =ε\emptyset^* = \varepsilon
      • R=RR^{**} = R^*
      • R=εRRR^* = \varepsilon | RR^*

      Limitations

      Not all languages are regular. For example:

      • {anbn:n0}\{a^n b^n : n \geq 0\} is NOT regular
      • {ww:wΣ}\{ww : w \in \Sigma^*\} is NOT regular
      • Balanced parentheses are NOT regular

      These require more expressive formalisms (context-free grammars, etc.).


      Summary

      • Regular expressions are built from atoms (ε\varepsilon, \emptyset, symbols) and constructors (union, concat, star)
      • Each regex denotes a language L(R)L(R)
      • Precedence: star > concat > union
      • Useful for pattern matching, lexical analysis
      • Not all languages are regular
    • Regular Expression Matching

      The Matching Relation

      Definition

      The matching relation (w,R)(w, R) means “string ww matches regex RR.”

      Equivalently: wL(R)w \in L(R).

      Inductive Definition of Matching

      Atoms

      (ε,Null)(a,Syma)\frac{}{(\varepsilon, \text{Null})} \qquad \frac{}{(a, \text{Sym}_a)}

      • The empty string matches ε\varepsilon
      • The symbol aa matches the regex aa

      Note: No string matches \emptyset (never).

      Union

      (u,R)(u,Union(R,S))(v,S)(v,Union(R,S))\frac{(u, R)}{(u, \text{Union}(R, S))} \qquad \frac{(v, S)}{(v, \text{Union}(R, S))}

      String uu matches RSR|S if uu matches RR OR uu matches SS.

      Concatenation

      (u,R)(v,S)(uv,Concat(R,S))\frac{(u, R) \quad (v, S)}{(uv, \text{Concat}(R, S))}

      String uvuv matches RSRS if uu matches RR and vv matches SS.

      Star

      (ε,Star(R))(u,R)(v,Star(R))(uv,Star(R))\frac{}{(\varepsilon, \text{Star}(R))} \qquad \frac{(u, R) \quad (v, \text{Star}(R))}{(uv, \text{Star}(R))}

      The empty string matches RR^*, and if uu matches RR and vv matches RR^*, then uvuv matches RR^*.

      Example Derivations

      Matching abab against abab

            (ab, Concat(a,b))
               /          \
          (a, a)        (b, b)

      Matching abab against aba^*b^*

      One possible derivation:

          (ab, Concat(Star(a), Star(b)))
                 /                    \
         (a, Star(a))            (b, Star(b))
            /     \                 /     \
      (a, a)  (ε, Star(a))    (b, b)  (ε, Star(b))

      Matching aaabaaab against aba^*b

            (aaab, Concat(Star(a), b))
                  /              \
          (aaa, Star(a))      (b, b)
              /      \
         (aa, Star(a))  (a, a)
            /      \
      (a, Star(a))  (a, a)
          |
      (ε, Star(a))

      Proving Properties by Rule Induction

      Example: Star Matches Repetitions

      Claim: ww matches RR^* iff w=w1w2wnw = w_1 w_2 \cdots w_n where each wiw_i matches RR.

      Proof:

      ()(\Rightarrow) Rule induction on matching derivation for (w,R)(w, R^*):

      • Base case: w=εw = \varepsilon = empty concatenation of RR-matches.
      • Inductive case: (uv,R)(uv, R^*) from (u,R)(u, R) and (v,R)(v, R^*). By IH, v=v1vmv = v_1 \cdots v_m with each vjv_j matching RR. So uv=uv1vmuv = u \cdot v_1 \cdots v_m where uu and each vjv_j match RR.

      ()(\Leftarrow) By induction on nn: if w=w1wnw = w_1 \cdots w_n with each wiw_i matching RR:

      • n=0n = 0: w=εw = \varepsilon matches RR^* by axiom.
      • n>0n > 0: w=w1wn1wnw = w_1 \cdots w_{n-1} \cdot w_n matches RR^* by concatenating matches.

      Non-Matching

      Establishing Non-Matching

      To prove ww does NOT match RR:

      1. Enumerate all possible derivations
      2. Show none succeed
      3. Or use meta-theorems about L(R)L(R)

      Example: baba does not match abab

      baba would need to match aba|b or concatenation of matches.

      • Not ε\varepsilon, not a single symbol matching aba|b
      • For concatenation: would need split ba=uvba = uv with uu matching aa and vv matching bb
        • u=bu = b: doesn’t match aa
        • u=bau = ba: v=εv = \varepsilon doesn’t match bb

      No derivation exists.

      Complexity of Matching

      Decision Problem

      Given ww and RR, does wL(R)w \in L(R)?

      Algorithms

      • Backtracking: Try all ways to match; exponential worst case
      • Thompson’s algorithm: Convert to NFA, simulate in O(wR)O(|w| \cdot |R|)
      • Glushkov’s algorithm: Direct NFA construction
      • DFA simulation: After NFA-to-DFA, O(w)O(|w|) matching

      Summary

      • Matching is defined inductively by rules
      • Union: match either branch
      • Concat: split and match both parts
      • Star: match zero or more repetitions
      • Non-matching established by showing no derivation exists
      • Efficient algorithms exist for regex matching
    • Parsing and Pretty Printing Regular Expressions

      Parsing: String to AST

      The Problem

      Given a concrete string like a|b*c, produce the abstract syntax tree.

      Precedence Resolution

      The string a|b*c should parse as a(bc)a|(b*c), not (ab)c(a|b)*c.

      This is implemented by a parsing relation with precedence levels.

      Parsing Relation

      Levels

      Define parsing at different precedence levels:

      • parseUnion: handles |
      • parseConcat: handles concatenation
      • parseStar: handles *
      • parseAtom: handles atoms and parentheses

      Grammar (Simplified)

      Union  ::= Concat ('|' Concat)*
      Concat ::= Star+
      Star   ::= Atom '*'*
      Atom   ::= ε | ∅ | symbol | '(' Union ')'

      Rules for Parsing

      Atoms: parseAtom(a,Syma)parseAtom(ε,Null)parseUnion(s,R)parseAtom((s),R)\frac{}{\text{parseAtom}(a, \text{Sym}_a)} \quad \frac{}{\text{parseAtom}(\varepsilon, \text{Null})} \quad \frac{\text{parseUnion}(s, R)}{\text{parseAtom}((s), R)}

      Star: parseAtom(s,R)parseStar(s,R)parseStar(s,R)parseStar(s,Star(R))\frac{\text{parseAtom}(s, R)}{\text{parseStar}(s, R)} \quad \frac{\text{parseStar}(s, R)}{\text{parseStar}(s*, \text{Star}(R))}

      Concatenation: parseStar(s,R)parseConcat(s,R)parseConcat(s1,R1)parseStar(s2,R2)parseConcat(s1s2,Concat(R1,R2))\frac{\text{parseStar}(s, R)}{\text{parseConcat}(s, R)} \quad \frac{\text{parseConcat}(s_1, R_1) \quad \text{parseStar}(s_2, R_2)}{\text{parseConcat}(s_1 s_2, \text{Concat}(R_1, R_2))}

      Union: parseConcat(s,R)parseUnion(s,R)parseUnion(s1,R1)parseConcat(s2,R2)parseUnion(s1s2,Union(R1,R2))\frac{\text{parseConcat}(s, R)}{\text{parseUnion}(s, R)} \quad \frac{\text{parseUnion}(s_1, R_1) \quad \text{parseConcat}(s_2, R_2)}{\text{parseUnion}(s_1|s_2, \text{Union}(R_1, R_2))}

      Example Parse

      String: a|b*

      1. Parse as Union
      2. Left part: a parses as Syma_a
      3. Right part: b*
        • b parses as Symb_b
        • b* parses as Star(Symb_b)
      4. Result: Union(Syma_a, Star(Symb_b))
          Union
          /   \
         a    Star
               |
               b

      Parentheses Disambiguate

      With Parentheses

      (a|b)*c parses as:

          Concat
          /    \
        Star    c
          |
        Union
        /   \
       a     b

      Without Parentheses

      a|b*c parses as a|(b*c):

          Union
          /   \
         a   Concat
              /   \
            Star   c
              |
              b

      Pretty Printing: AST to String

      The Problem

      Given an AST, produce a readable concrete string.

      Strategy

      1. Recursively convert subtrees to strings
      2. Add parentheses when child has lower precedence than parent
      3. Minimize parentheses for readability

      Precedence Table

      ConstructorPrecedence
      Star3 (highest)
      Concat2
      Union1 (lowest)

      Rules

      pprint(R,p)=pretty print R at precedence level p\text{pprint}(R, p) = \text{pretty print } R \text{ at precedence level } p

      Add parentheses if p>prec(R)p > \text{prec}(R).

      Atoms: Never need parentheses around themselves.

      Star: pprint(R,p)=pprint(R,3)+\text{pprint}(R^*, p) = \text{pprint}(R, 3) + *

      Concat: pprint(RS,p)=pprint(R,2)+pprint(S,2)\text{pprint}(RS, p) = \text{pprint}(R, 2) + \text{pprint}(S, 2), parenthesise if p>2p > 2.

      Union: pprint(RS,p)=pprint(R,1)++pprint(S,1)\text{pprint}(R|S, p) = \text{pprint}(R, 1) + | + \text{pprint}(S, 1), parenthesise if p>1p > 1.

      Example Pretty Prints

      Simple

      • pprint(a)=a\text{pprint}(a) = a
      • pprint(a)=a\text{pprint}(a^*) = a*
      • pprint(ab)=ab\text{pprint}(ab) = ab
      • pprint(ab)=ab\text{pprint}(a|b) = a|b

      With Parentheses

      • pprint((ab))=(ab)\text{pprint}((a|b)^*) = (a|b)* (union under star needs parens)
      • pprint(ab)=ab\text{pprint}(a^*b^*) = a*b* (no parens needed)
      • pprint((ab)(cd))=(ab)(cd)\text{pprint}((a|b)(c|d)) = (a|b)(c|d) (unions in concat need parens)

      Round-Trip Property

      Parsing then Pretty Printing

      For any string ss that parses to AST RR:

      parse(s)=Rpprint(R)=s\text{parse}(s) = R \Rightarrow \text{pprint}(R) = s'

      where ss' is semantically equivalent and may be ss or a normalised form.

      Pretty Printing then Parsing

      For any AST RR:

      parse(pprint(R))=R\text{parse}(\text{pprint}(R)) = R

      The pretty-printed string should re-parse to the same AST.


      Summary

      • Parsing converts concrete strings to abstract syntax trees
      • Precedence determines how strings are structured into trees
      • Pretty printing converts ASTs back to readable strings
      • Parentheses are added only when necessary for correctness
    • Pattern Matching: From Rule Induction to Functional Programming

      The Connection

      Rule induction in discrete maths and pattern matching in functional programming (OCaml, Haskell) are deeply connected. Both deal with inductively defined structures and recursive processing.


      Recap: Inductive Definitions

      A set defined inductively specifies:

      • Axioms: base elements
      • Rules: ways to construct new elements from existing ones

      Example: Arithmetic Expressions

      E::=Num(n)Add(E,E)Mul(E,E)E ::= \text{Num}(n) \mid \text{Add}(E, E) \mid \text{Mul}(E, E)

      The set of expressions includes:

      • Num(5)\text{Num}(5)
      • Add(Num(3),Num(7))\text{Add}(\text{Num}(3), \text{Num}(7))
      • Mul(Add(Num(1),Num(2)),Num(3))\text{Mul}(\text{Add}(\text{Num}(1), \text{Num}(2)), \text{Num}(3))

      Abstract syntax tree for expressions


      Abstract Syntax Trees

      Concrete Syntax

      The textual representation: 1 + 2 * 3

      Abstract Syntax Tree (AST)

      The tree structure:

            Add
           /   \
        Num(1)  Mul
               /   \
            Num(2) Num(3)

      OCaml Data Type

      The BNF grammar translates directly to an algebraic data type:

      type expr =
        | Num of int
        | Add of expr * expr
        | Mul of expr * expr

      Each constructor corresponds to a rule in the inductive definition.


      Rule Induction Recap

      The Principle

      To prove a property P(e)P(e) holds for all expressions ee:

      1. Base case (axioms): Prove P(Num(n))P(\text{Num}(n)) for all nn
      2. Inductive step (rules): Assuming P(e1)P(e_1) and P(e2)P(e_2), prove P(Add(e1,e2))P(\text{Add}(e_1, e_2)) and P(Mul(e1,e2))P(\text{Mul}(e_1, e_2))

      Example: All Expressions Have Non-negative Size

      Define size(e)\text{size}(e) as the number of constructors:

      size(Num(n))=1\text{size}(\text{Num}(n)) = 1 size(Add(e1,e2))=1+size(e1)+size(e2)\text{size}(\text{Add}(e_1, e_2)) = 1 + \text{size}(e_1) + \text{size}(e_2) size(Mul(e1,e2))=1+size(e1)+size(e2)\text{size}(\text{Mul}(e_1, e_2)) = 1 + \text{size}(e_1) + \text{size}(e_2)

      By rule induction: size(e)1\text{size}(e) \geq 1 for all ee.


      Pattern Matching in OCaml

      The Core Idea

      A match expression examines the structure of data and:

      1. Identifies which constructor was used
      2. Extracts (binds) the sub-components to variables

      Evaluating Expressions

      let rec eval (e : expr) : int =
        match e with
        | Num n -> n
        | Add (e1, e2) -> eval e1 + eval e2
        | Mul (e1, e2) -> eval e1 * eval e2

      Correspondence to Rule Induction

      Pattern match branchRule induction step
      `Num n -> …`
      `Add (e1, e2) -> …`
      `Mul (e1, e2) -> …`

      The recursive calls eval e1 and eval e2 apply the function to the sub-expressions, exactly as rule induction applies the hypothesis to premises.


      Variable Binding

      The Mechanism

      When pattern matching encounters:

      | Add (e1, e2) -> ...

      and the input is Add (Num 3, Num 7):

      1. The constructor Add matches
      2. e1 is bound to Num 3
      3. e2 is bound to Num 7
      4. The right-hand side executes with these bindings

      Pattern matching and variable binding


      Exhaustiveness Checking

      The Compiler’s Job

      The OCaml compiler checks that a match covers ALL constructors.

      let rec eval (e : expr) : int =
        match e with
        | Num n -> n
        | Add (e1, e2) -> eval e1 + eval e2
      (* Warning: this pattern-matching is not exhaustive. *)

      The compiler warns that Mul is missing.

      Why This Works

      Because the data type was inductively defined with a fixed set of constructors, the compiler knows exactly what patterns are possible.

      Mathematical Connection

      Exhaustiveness checking is the compiler verifying that your proof covers all axioms and rules!


      Structural Recursion

      Definition

      A function is structurally recursive if:

      • Each recursive call is on a proper sub-structure of the input
      • There is a base case for each axiom

      This guarantees termination.

      Example: Size Function

      let rec size (e : expr) : int =
        match e with
        | Num _ -> 1
        | Add (e1, e2) -> 1 + size e1 + size e2
        | Mul (e1, e2) -> 1 + size e1 + size e2

      The recursion follows the tree structure:

      • Base case: Num _ has size 1
      • Recursive calls on e1 and e2 are on smaller sub-trees

      Termination

      Since the AST has finite depth, structural recursion always terminates.


      Example: Pretty Printing

      Convert an AST back to a string:

      let rec pretty (e : expr) : string =
        match e with
        | Num n -> string_of_int n
        | Add (e1, e2) -> 
            "(" ^ pretty e1 ^ " + " ^ pretty e2 ^ ")"
        | Mul (e1, e2) ->
            "(" ^ pretty e1 ^ " * " ^ pretty e2 ^ ")"

      Pattern: recurse on sub-expressions, combine results.


      Example: Optimisation

      Simplify expressions (constant folding):

      let rec simplify (e : expr) : expr =
        match e with
        | Num n -> Num n
        | Add (e1, e2) ->
            (match simplify e1, simplify e2 with
             | Num 0, e -> e          (* 0 + e = e *)
             | e, Num 0 -> e          (* e + 0 = e *)
             | Num n1, Num n2 -> Num (n1 + n2)  (* fold *)
             | e1', e2' -> Add (e1', e2'))
        | Mul (e1, e2) ->
            (match simplify e1, simplify e2 with
             | Num 0, _ -> Num 0
             | _, Num 0 -> Num 0
             | Num 1, e -> e
             | e, Num 1 -> e
             | Num n1, Num n2 -> Num (n1 * n2)
             | e1', e2' -> Mul (e1', e2'))

      Lists: Another Inductive Structure

      Definition

      List(A)::=NilCons(A,List(A))\text{List}(A) ::= \text{Nil} \mid \text{Cons}(A, \text{List}(A))

      OCaml Type

      type 'a list =
        | []
        | (::) of 'a * 'a list

      Pattern Matching

      let rec length (lst : 'a list) : int =
        match lst with
        | [] -> 0
        | _ :: xs -> 1 + length xs

      Base case: empty list. Inductive step: head + tail.


      Summary

      Discrete MathsFunctional Programming
      Inductive definitionAlgebraic data type
      AxiomConstructor (e.g., Num)
      RuleConstructor with arguments (e.g., Add)
      Rule induction proofRecursive function
      Base caseBase case pattern
      Inductive hypothesisRecursive call
      Variable in rulePattern variable
      Exhaustive cases?Exhaustiveness check

      Pattern matching is executable rule induction: the compiler verifies your proof structure, and the machine executes the computation.


      Tripos Questions

      2021 P1 Q4: Explain how pattern matching in functional programming relates to proof by rule induction.

      Paper 1 FOCS: Implement a recursive function to compute the depth of an expression tree.

  • Automata Theory

    NFAs, DFAs, subset construction, Kleene's theorem, Thompson's construction, and the pumping lemma

    • NFA: Nondeterministic Finite Automata

      Definition

      A nondeterministic finite automaton (NFA) is a tuple M=(Q,Σ,Δ,s,F)M = (Q, \Sigma, \Delta, s, F) where:

      ComponentDescription
      QQFinite set of states
      Σ\SigmaInput alphabet (finite)
      ΔQ×Σ×Q\Delta \subseteq Q \times \Sigma \times QTransition relation
      sQs \in QStart state
      FQF \subseteq QSet of accepting states

      Key Feature: Nondeterminism

      From a state qq on input aa, there may be:

      • Zero transitions: no next state
      • One transition: one next state (like DFA)
      • Multiple transitions: several possible next states

      NFA example with transitions

      Extended Transition Relation

      One-Step Transition

      Write qaqq \xrightarrow{a} q' if (q,a,q)Δ(q, a, q') \in \Delta.

      Multi-Step Transition

      Define qwqq \xrightarrow{w} q' (there is a path from qq to qq' labelled by ww) inductively:

      qεqqaqqwqqawq\frac{}{q \xrightarrow{\varepsilon} q} \qquad \frac{q \xrightarrow{a} q'' \quad q'' \xrightarrow{w} q'}{q \xrightarrow{aw} q'}

      Language

      The NFA accepts string ww if there exists an accepting state reachable from the start:

      wL(M)    qfF.swqfw \in L(M) \iff \exists q_f \in F. s \xrightarrow{w} q_f

      Example: Contains “ab”

      NFA that accepts strings over {a,b}\{a, b\} containing substring “ab”:

      NFA for "contains ab"

      • Q={q0,q1,q2}Q = \{q_0, q_1, q_2\}
      • Transitions:
        • q0aq0q_0 \xrightarrow{a} q_0, q0bq0q_0 \xrightarrow{b} q_0 (stay in q0q_0 until we see ‘a’)
        • q0aq1q_0 \xrightarrow{a} q_1 (transition to q1q_1 if we see ‘a’)
        • q1bq2q_1 \xrightarrow{b} q_2 (transition to accepting on ‘b’)
        • q2aq2q_2 \xrightarrow{a} q_2, q2bq2q_2 \xrightarrow{b} q_2 (stay accepting)
      • s=q0s = q_0, F={q2}F = \{q_2\}

      Example: Ends with “ab”

      NFA that accepts strings ending in “ab”:

      NFA for "ends with ab"

      • Q={q0,q1,q2}Q = \{q_0, q_1, q_2\}
      • Transitions:
        • q0aq0q_0 \xrightarrow{a} q_0, q0bq1q_0 \xrightarrow{b} q_1
        • q1aq2q_1 \xrightarrow{a} q_2, q1bq1q_1 \xrightarrow{b} q_1
        • q2aq0q_2 \xrightarrow{a} q_0, q2bq1q_2 \xrightarrow{b} q_1
      • Or: more standard construction with q0a,bq0q_0 \xrightarrow{a,b} q_0, plus q0aq1q_0 \xrightarrow{a} q_1, q1bq2q_1 \xrightarrow{b} q_2
      • s=q0s = q_0, F={q2}F = \{q_2\}

      Non-acceptance

      An NFA rejects string ww if:

      • No path from ss reaches any accepting state on input ww, OR
      • The computation “gets stuck” (no valid transition)

      Note: Nondeterminism means existential acceptance — if ANY path reaches accepting, the string is accepted.

      Size and Complexity

      • Number of states: Q|Q|
      • Number of transitions: Δ|\Delta|
      • For each state-symbol pair, potentially Q|Q| outgoing transitions

      Summary

      • NFA: (Q,Σ,Δ,s,F)(Q, \Sigma, \Delta, s, F) with nondeterministic transition relation
      • A string is accepted if SOME path reaches an accepting state
      • Multiple transitions from a state on the same symbol allowed
      • No transition on a symbol means that path dies
    • NFA with Epsilon Transitions

      Motivation

      When combining NFAs, it is useful to have “silent” transitions that consume no input.

      Definition

      An NFA-ε\varepsilon is an NFA with an additional ε\varepsilon-transition relation τQ×Q\tau \subseteq Q \times Q.

      We write qεqq \xrightarrow{\varepsilon} q' if (q,q)τ(q, q') \in \tau.

      Full Specification

      M=(Q,Σ,Δ,τ,s,F)M = (Q, \Sigma, \Delta, \tau, s, F) where:

      • Q,Σ,s,FQ, \Sigma, s, F as before
      • ΔQ×Σ×Q\Delta \subseteq Q \times \Sigma \times Q: labelled transitions
      • τQ×Q\tau \subseteq Q \times Q: ε\varepsilon-transitions

      Extended Transition Relation

      Epsilon Closure

      The ε\varepsilon-closure of state qq, written E(q)E(q), is the set of states reachable from qq via only ε\varepsilon-transitions:

      E(q)={q:qεq}E(q) = \{q' : q \xrightarrow{\varepsilon^*} q'\}

      where ε\xrightarrow{\varepsilon^*} is the reflexive-transitive closure of ε\xrightarrow{\varepsilon}.

      Extended Transition

      Define qwqq \Rightarrow w q' inductively:

      qεqqaqqwqqawqqεqqwqqwq\frac{}{q \Rightarrow_\varepsilon q} \qquad \frac{q \xrightarrow{a} q'' \quad q'' \Rightarrow_w q'}{q \Rightarrow_{aw} q'} \qquad \frac{q \xrightarrow{\varepsilon} q'' \quad q'' \Rightarrow_w q'}{q \Rightarrow_w q'}

      Language

      wL(M)    qfF.swqfw \in L(M) \iff \exists q_f \in F. s \Rightarrow_w q_f

      Example

      NFA-ε\varepsilon for aba^*b:

      NFA-epsilon for a*b

      • Q={q0,q1,q2}Q = \{q_0, q_1, q_2\}
      • q0aq0q_0 \xrightarrow{a} q_0 (loop on ‘a’)
      • q0εq1q_0 \xrightarrow{\varepsilon} q_1 (epsilon transition)
      • q1bq2q_1 \xrightarrow{b} q_2 (accept on ‘b’)
      • F={q2}F = \{q_2\}

      Accepts: bb, abab, aabaab, aaabaaab, … (strings of aa‘s ending in bb)

      Epsilon Closure Example

      For the above NFA:

      • E(q0)={q0,q1}E(q_0) = \{q_0, q_1\} (can reach q1q_1 via ε\varepsilon)
      • E(q1)={q1}E(q_1) = \{q_1\}
      • E(q2)={q2}E(q_2) = \{q_2\}

      Eliminating Epsilon Transitions

      Any NFA-ε\varepsilon can be converted to an equivalent NFA without ε\varepsilon-transitions.

      Construction

      Given M=(Q,Σ,Δ,τ,s,F)M = (Q, \Sigma, \Delta, \tau, s, F), define NFA M=(Q,Σ,Δ,s,F)M' = (Q, \Sigma, \Delta', s', F'):

      Δ={(q,a,q):q.qεqqaq}\Delta' = \{(q, a, q') : \exists q''. q \xrightarrow{\varepsilon^*} q'' \land q'' \xrightarrow{a} q'\}

      s=ss' = s F=F{q:E(q)F}F' = F \cup \{q : E(q) \cap F \neq \emptyset\}

      Using Epsilon Transitions for Concatenation

      To recognise L1L2L_1 \cdot L_2, connect NFAs:

      1. Add ε\varepsilon-transitions from accepting states of M1M_1 to start of M2M_2
      2. Start state is s1s_1
      3. Accepting states are those of M2M_2

      Concatenation via epsilon transitions

      Using Epsilon for Union

      To recognise L1L2L_1 \cup L_2:

      1. Create new start state snews_{\text{new}}
      2. Add ε\varepsilon-transitions to s1s_1 and s2s_2
      3. Accepting states are F1F2F_1 \cup F_2

      Union via epsilon transitions


      Summary

      • NFA-ε\varepsilon allows transitions that consume no input
      • E(q)E(q) = set of states reachable via ε\varepsilon-transitions only
      • ε\varepsilon-transitions simplify construction of combined automata
      • Any NFA-ε\varepsilon can be converted to equivalent NFA
    • DFA: Deterministic Finite Automata

      Definition

      A deterministic finite automaton (DFA) is a tuple M=(Q,Σ,δ,s,F)M = (Q, \Sigma, \delta, s, F) where:

      ComponentDescription
      QQFinite set of states
      Σ\SigmaInput alphabet (finite)
      δ:Q×ΣQ\delta : Q \times \Sigma \to QTransition function
      sQs \in QStart state
      FQF \subseteq QAccepting states

      Key Feature: Determinism

      For every state qq and symbol aa, there is exactly one next state δ(q,a)\delta(q, a).

      • No ambiguity
      • No “stuck” states (unless δ\delta is partial)
      • Exactly one computation path for each input

      Extended Transition Function

      Definition

      Extend δ:Q×ΣQ\delta : Q \times \Sigma \to Q to δ^:Q×ΣQ\hat{\delta} : Q \times \Sigma^* \to Q:

      δ^(q,ε)=q\hat{\delta}(q, \varepsilon) = q δ^(q,aw)=δ^(δ(q,a),w)\hat{\delta}(q, aw) = \hat{\delta}(\delta(q, a), w)

      Language

      wL(M)    δ^(s,w)Fw \in L(M) \iff \hat{\delta}(s, w) \in F

      The string is accepted iff the unique computation path ends in an accepting state.

      Example: Divisible by 3

      DFA over {0,1,2}\{0, 1, 2\} accepting numbers whose digit sum is divisible by 3:

      DFA for divisible by 3

      • Q={q0,q1,q2}Q = \{q_0, q_1, q_2\} (remainder mod 3)
      • δ(qi,d)=q(i+d)mod3\delta(q_i, d) = q_{(i+d) \bmod 3}
      • s=q0s = q_0, F={q0}F = \{q_0\}

      Example: Even Parity

      DFA over {0,1}\{0, 1\} accepting strings with an even number of 1s:

      DFA for even parity

      • Q={q0,q1}Q = \{q_0, q_1\} (parity of seen 1s)
      • δ(q0,0)=q0\delta(q_0, 0) = q_0, δ(q0,1)=q1\delta(q_0, 1) = q_1
      • δ(q1,0)=q1\delta(q_1, 0) = q_1, δ(q1,1)=q0\delta(q_1, 1) = q_0
      • s=q0s = q_0, F={q0}F = \{q_0\}

      Example: Ends with “ab”

      DFA over {a,b}\{a, b\} accepting strings ending in “ab”:

      DFA for ends with ab

      • Q={q0,q1,q2,qd}Q = \{q_0, q_1, q_2, q_d\}
      • Transitions (key ones):
        • q0q_0: last char is bb (or start)
        • q1q_1: last char is aa
        • q2q_2: last two chars are abab
        • qdq_d: dead state (or integrate below)
      • More commonly: Q={q0,q1,q2}Q = \{q_0, q_1, q_2\} with careful transitions
      StateInput aaInput bb
      q0q_0q1q_1q0q_0
      q1q_1q1q_1q2q_2
      q2q_2q1q_1q0q_0

      F={q2}F = \{q_2\}

      Complete vs Partial DFAs

      Complete DFA

      δ\delta is total: there is exactly one transition for every (q,a)Q×Σ(q, a) \in Q \times \Sigma.

      Every input has a unique complete computation.

      Partial DFA

      δ\delta may be undefined for some (q,a)(q, a).

      If computation reaches an undefined transition, the string is rejected.

      Conversion

      Any partial DFA can be converted to a complete DFA by adding a “trap” or “dead” state.

      Rejection

      A DFA rejects string ww if:

      δ^(s,w)F\hat{\delta}(s, w) \notin F

      The computation exists (deterministically) but ends in a non-accepting state.

      State Diagram Convention

      • Circles represent states
      • Double circles represent accepting states
      • Arrow from “start” marks the start state
      • Labelled arrows between states show transitions

      Summary

      • DFA: (Q,Σ,δ,s,F)(Q, \Sigma, \delta, s, F) with deterministic transition function
      • Exactly one computation path per input
      • Accept iff final state is in FF
      • Complete vs partial: complete has all transitions defined
    • NFA and DFA Examples

      Example 1: Contains “aaa”

      NFA

      NFA accepting strings over {a,b}\{a, b\} containing three consecutive aa‘s:

      NFA for contains aaa

      • Q={q0,q1,q2,q3}Q = \{q_0, q_1, q_2, q_3\}
      • Transitions:
        • q0a,bq0q_0 \xrightarrow{a,b} q_0 (stay until we start counting)
        • q0aq1q_0 \xrightarrow{a} q_1 (first aa)
        • q1aq2q_1 \xrightarrow{a} q_2 (second aa)
        • q2aq3q_2 \xrightarrow{a} q_3 (third aa, accept)
        • q3a,bq3q_3 \xrightarrow{a,b} q_3 (stay accepting)
      • F={q3}F = \{q_3\}

      DFA Equivalent

      Via subset construction, the DFA has up to 24=162^4 = 16 states, but many are unreachable.

      Example 2: Contains “ab” as Substring

      NFA

      NFA for contains ab

      Intuitively: stay in q0q_0, transition to q1q_1 on aa, then to q2q_2 on bb, stay in q2q_2.

      DFA

      DFA for contains ab

      • Q={q0,q1,q2}Q = \{q_0, q_1, q_2\}
      • Transitions:
      Stateaabb
      q0q_0q1q_1q0q_0
      q1q_1q1q_1q2q_2
      q2q_2q2q_2q2q_2

      F={q2}F = \{q_2\}

      Example 3: Modulo Counter

      DFA: Divisible by 4 (Binary)

      Accepts binary strings divisible by 4:

      DFA for divisible by 4

      • Q={q0,q1,q2,q3}Q = \{q_0, q_1, q_2, q_3\} (remainder mod 4)
      • δ(qr,0)=q(2r)mod4\delta(q_r, 0) = q_{(2r) \bmod 4}
      • δ(qr,1)=q(2r+1)mod4\delta(q_r, 1) = q_{(2r+1) \bmod 4}
      • F={q0}F = \{q_0\}

      Intuition

      Reading binary left-to-right:

      • Append 0: multiply by 2
      • Append 1: multiply by 2, add 1

      Example 4: Exact Pattern

      DFA: Exactly “ab”

      DFA for exactly ab

      • Q={q0,q1,q2,qd}Q = \{q_0, q_1, q_2, q_d\}
      • Transitions:
      Stateaabb
      q0q_0q1q_1qdq_d
      q1q_1qdq_dq2q_2
      q2q_2qdq_dqdq_d
      qdq_dqdq_dqdq_d

      F={q2}F = \{q_2\}

      Dead state qdq_d ensures no other strings are accepted.

      Example 5: At Most Two aa‘s

      DFA

      Accepts strings with at most two aa‘s:

      • Q={q0,q1,q2,q3}Q = \{q_0, q_1, q_2, q_3\}
      • States track how many aa‘s seen so far
      • qiq_i: seen exactly ii aa‘s
      • q3q_3: seen three or more aa‘s (rejecting trap)
      Stateaabb
      q0q_0q1q_1q0q_0
      q1q_1q2q_2q1q_1
      q2q_2q3q_3q2q_2
      q3q_3q3q_3q3q_3

      F={q0,q1,q2}F = \{q_0, q_1, q_2\}

      Example 6: Union via NFA-ε\varepsilon

      Languages

      L1={w:w ends in a}L_1 = \{w : w \text{ ends in } a\}

      L2={w:w ends in b}L_2 = \{w : w \text{ ends in } b\}

      NFA-ε\varepsilon for L1L2L_1 \cup L_2

      Create new start with ε\varepsilon-transitions to both machines:

      Union NFA-epsilon example

      Accepting: both F1F_1 and F2F_2.


      Summary

      • NFAs are easier to construct for pattern detection
      • DFAs are canonical (unique minimal DFA)
      • Union uses epsilon transitions in NFAs
      • Counting/modulo uses state to track remainder
    • The Subset Construction

      The Goal

      Convert an NFA (or NFA-ε\varepsilon) MM to an equivalent DFA P(M)\mathcal{P}(M) that recognises the same language.

      The Construction

      Input

      NFA-ε\varepsilon M=(Q,Σ,Δ,τ,s,F)M = (Q, \Sigma, \Delta, \tau, s, F)

      Output

      DFA P(M)=(Q,Σ,δ,s,F)\mathcal{P}(M) = (Q', \Sigma, \delta', s', F')

      States

      Q=P(Q)={S:SQ}Q' = \mathcal{P}(Q) = \{S : S \subseteq Q\}

      Each DFA state is a set of NFA states.

      Start State

      s=E(s)={q:sεq}s' = E(s) = \{q' : s \xrightarrow{\varepsilon^*} q'\}

      The ε\varepsilon-closure of the NFA’s start state.

      Accepting States

      F={SQ:SF}F' = \{S \subseteq Q : S \cap F \neq \emptyset\}

      A DFA state is accepting if it contains at least one NFA accepting state.

      Transition Function

      For SQS \subseteq Q and aΣa \in \Sigma:

      δ(S,a)=qS{q:qaqεq for some q}\delta'(S, a) = \bigcup_{q \in S} \{q' : q \xrightarrow{a} q'' \xrightarrow{\varepsilon^*} q' \text{ for some } q''\}

      Equivalently: the set of all states reachable from SS on symbol aa, followed by any number of ε\varepsilon-transitions.

      Algorithm Summary

      1. Start with ε\varepsilon-closure of {s}\{s\}
      2. For each reachable set SS and symbol aa, compute δ(S,a)\delta'(S, a)
      3. Add newly discovered states to worklist
      4. Repeat until no new states

      Example

      NFA

      NFA for subset construction example

      MM: Q={q0,q1,q2}Q = \{q_0, q_1, q_2\}, s=q0s = q_0, F={q2}F = \{q_2\}

      Transitions:

      • q0aq0q_0 \xrightarrow{a} q_0, q0aq1q_0 \xrightarrow{a} q_1
      • q1bq2q_1 \xrightarrow{b} q_2

      DFA Construction

      Start: s=E(q0)={q0}s' = E(q_0) = \{q_0\}

      Compute transitions:

      DFA Stateaabb
      {q0}\{q_0\}{q0,q1}\{q_0, q_1\}\emptyset
      {q0,q1}\{q_0, q_1\}{q0,q1}\{q_0, q_1\}{q2}\{q_2\}
      {q2}\{q_2\}\emptyset\emptyset
      \emptyset\emptyset\emptyset

      Accepting: F={{q2}}F' = \{\{q_2\}\} (states containing q2q_2)

      Resulting DFA

      DFA from subset construction

      Size

      Worst Case

      The DFA can have up to 2Q2^{|Q|} states.

      Typical Case

      Many subsets may be unreachable, so the actual DFA is often smaller.

      Exponential Blowup

      Some NFAs require exponentially many DFA states. Example: language of strings where the nn-th symbol from the end is aa requires Ω(2n)\Omega(2^n) DFA states but only n+1n+1 NFA states.

      Key Intuition

      The DFA simulates “all possible computations” of the NFA simultaneously.

      • DFA state == set of NFA states that could be active
      • Transition == move all active states, take ε\varepsilon-closures
      • Accepting == any active state is an NFA accept state

      Summary

      • Subset construction: NFA state \mapsto DFA state = set of NFA states
      • DFA simulates all NFA computations in parallel
      • Start state = ε\varepsilon-closure of NFA start
      • Accept if intersection with FF is non-empty
      • Worst case: exponential blowup in state count
    • Correctness of the Subset Construction

      Statement

      Theorem: Let MM be an NFA-ε\varepsilon and P(M)\mathcal{P}(M) the result of subset construction. Then L(M)=L(P(M))L(M) = L(\mathcal{P}(M)).

      We prove two directions:

      1. Soundness: L(P(M))L(M)L(\mathcal{P}(M)) \subseteq L(M)
      2. Completeness: L(M)L(P(M))L(M) \subseteq L(\mathcal{P}(M))

      Examinable Proof: Soundness

      Claim: If P(M)\mathcal{P}(M) accepts ww, then MM accepts ww.

      Proof:

      Suppose P(M)\mathcal{P}(M) accepts w=a1a2anw = a_1 a_2 \cdots a_n.

      There is a DFA computation:

      sa1S1a2anSns' \xrightarrow{a_1} S_1 \xrightarrow{a_2} \cdots \xrightarrow{a_n} S_n

      where SnFS_n \cap F \neq \emptyset.

      Build an NFA computation:

      We show by induction on ii that for each SiS_i, and for each qSiq \in S_i, there is an NFA path from ss to qq on input a1aia_1 \cdots a_i.

      Base (i=0i = 0): s=E(s)s' = E(s). Every qsq \in s' is reachable from ss via ε\varepsilon^*.

      Step: Suppose for each qSi1q' \in S_{i-1}, there is an NFA path sa1ai1qs \Rightarrow a_1 \cdots a_{i-1} q'.

      For qSiq \in S_i: By construction, qδ(Si1,ai)q \in \delta'(S_{i-1}, a_i).

      So there exists qSi1q' \in S_{i-1} with qaiqεqq' \xrightarrow{a_i} q'' \xrightarrow{\varepsilon^*} q for some qq''.

      By IH, sa1ai1qs \Rightarrow a_1 \cdots a_{i-1} q'. Thus sa1aiqs \Rightarrow a_1 \cdots a_i q.

      Conclusion: Since SnFS_n \cap F \neq \emptyset, there exists qfFSnq_f \in F \cap S_n reachable from ss via ww. Hence MM accepts ww.

      Examinable Proof: Completeness

      Claim: If MM accepts ww, then P(M)\mathcal{P}(M) accepts ww.

      Proof:

      Suppose MM accepts w=a1anw = a_1 \cdots a_n.

      There exists an NFA computation:

      s=q0a1q1a2anqnFs = q_0 \xrightarrow{a_1} q_1 \xrightarrow{a_2} \cdots \xrightarrow{a_n} q_n \in F

      (where each step may involve ε\varepsilon-transitions).

      Build the DFA computation:

      Define DFA states S0,S1,,SnS_0, S_1, \ldots, S_n where Si=S_i = the set of NFA states reachable from ss on input a1aia_1 \cdots a_i via any path.

      Formally: Si=δ(Si1,ai)S_i = \delta'(S_{i-1}, a_i) with S0=E(s)=sS_0 = E(s) = s'.

      Key lemma: By construction, qiSiq_i \in S_i for all ii.

      Proof of lemma: By induction on ii.

      • q0E(s)=S0q_0 \in E(s) = S_0 (by definition of NFA computation starting at ss).
      • If qi1Si1q_{i-1} \in S_{i-1} and qi1aiqiq_{i-1} \xrightarrow{a_i} q_i, then qiδ(Si1,ai)=Siq_i \in \delta'(S_{i-1}, a_i) = S_i.

      Conclusion: qnSnq_n \in S_n. Since qnFq_n \in F, we have SnFS_n \cap F \neq \emptyset. Thus P(M)\mathcal{P}(M) accepts ww.

      Equivalence of NFA, NFA-ε\varepsilon, and DFA

      Corollary

      (NFA languages)=(NFA-ε languages)=(DFA languages)(\text{NFA languages}) = (\text{NFA-}\varepsilon\text{ languages}) = (\text{DFA languages})

      All three formalisms recognise exactly the same class of languages: the regular languages.

      Proof

      • NFA subset of NFA-ε\varepsilon: trivial (NFA is NFA-ε\varepsilon with empty τ\tau)
      • NFA-ε\varepsilon subset of DFA: subset construction (theorem above)
      • DFA subset of NFA: trivial (DFA is special case of NFA)

      Summary

      • Soundness: DFA path gives NFA path by tracking which NFA states are reachable
      • Completeness: NFA path traced through DFA by tracking sets
      • All three automata types (NFA, NFA-ε\varepsilon, DFA) recognise the same languages
    • Equivalence of NFA and DFA

      Main Theorem

      Theorem: The class of languages recognised by NFAs equals the class recognised by DFAs.

      L is NFA-recognisable    L is DFA-recognisableL \text{ is NFA-recognisable} \iff L \text{ is DFA-recognisable}

      Proof Outline

      DFA \Rightarrow NFA

      Every DFA is trivially an NFA where each state-symbol pair has exactly one transition.

      NFA \Rightarrow DFA

      The subset construction converts any NFA to an equivalent DFA.

      Since subset construction preserves language, any NFA-recognisable language is DFA-recognisable.

      Extending to NFA-ε\varepsilon

      NFA-ε\varepsilon \Rightarrow NFA

      Given NFA-ε\varepsilon M=(Q,Σ,Δ,τ,s,F)M = (Q, \Sigma, \Delta, \tau, s, F):

      Construct NFA M=(Q,Σ,Δ,s,F)M' = (Q, \Sigma, \Delta', s, F') where:

      Δ={(q,a,q):q.qεq(q,a,q)Δ}\Delta' = \{(q, a, q') : \exists q''. q \xrightarrow{\varepsilon^*} q'' \land (q'', a, q') \in \Delta\}

      F=F{q:E(q)F}F' = F \cup \{q : E(q) \cap F \neq \emptyset\}

      Intuitively: absorb ε\varepsilon-transitions into labelled transitions.

      Proof of Correctness

      The extended transition relation is preserved:

      • In MM: qwqq \Rightarrow_w q' via labelled and ε\varepsilon-transitions
      • In MM': qwqq \xrightarrow{w} q' via labelled transitions only

      Complete Equivalence

      All three models recognise the same languages:

      DFANFANFA-ε\text{DFA} \subset \text{NFA} \subset \text{NFA-}\varepsilon

      where each \subset is strict in expressive convenience but equal in language-theoretic power.

      The Class: Regular Languages

      Definition

      A language LL is regular if there exists a finite automaton (DFA, NFA, or NFA-ε\varepsilon) that recognises it.

      Equivalent Characterisations

      LL is regular iff any of:

      1. L=L(M)L = L(M) for some DFA MM
      2. L=L(M)L = L(M) for some NFA MM
      3. L=L(M)L = L(M) for some NFA-ε\varepsilon MM
      4. L=L(R)L = L(R) for some regular expression RR (Kleene’s theorem)
      5. LL is generated by a regular grammar
      6. LL is the language of a syntactic monoid

      Practical Implications

      For Design

      • NFAs are often easier to design (less constrained)
      • NFA-ε\varepsilon enables modular composition
      • DFAs have unique minimal form (useful for optimisation)

      For Implementation

      • DFAs run in O(w)O(|w|) time
      • NFAs can be simulated in O(wQ)O(|w| \cdot |Q|) time
      • Subset construction may cause exponential blowup

      For Theory

      • Closure properties are easier to prove with NFAs
      • Decidability results apply to all three

      Closure Properties

      Regular languages are closed under:

      • Union: disjoint union or NFA-ε\varepsilon construction
      • Concatenation: sequential composition
      • Kleene star: loop back construction
      • Complement: swap accepting/non-accepting in DFA
      • Intersection: product construction on DFAs

      Summary

      • DFA, NFA, and NFA-ε\varepsilon are equally expressive
      • All recognise exactly the regular languages
      • Subset construction: NFA \to DFA
      • Epsilon elimination: NFA-ε\varepsilon \to NFA
      • Each model has practical advantages
    • Kleene's Theorem

      Statement

      Kleene’s Theorem: A language LΣL \subseteq \Sigma^* is regular (recognised by a finite automaton) if and only if it is denoted by a regular expression.

      L is regular     regex R.L=L(R)L \text{ is regular} \iff \exists \text{ regex } R. L = L(R)

      Two Directions

      Part (a): Regex \Rightarrow Automaton

      Every regular expression can be converted to an equivalent NFA-ε\varepsilon.

      Part (b): Automaton \Rightarrow Regex

      Every finite automaton can be converted to an equivalent regular expression.

      Part (a): Regex to NFA

      Construction by Induction

      Base cases:

      ε\varepsilon: NFA with one state, start = accept, no transitions.

      q\xrightarrow{\quad} \textcircled{\scriptstyle q}

      \emptyset: NFA with one state, start state, no accepting states.

      q\xrightarrow{\quad} q

      Symbol aa: NFA with two states.

      q0aq1\xrightarrow{\quad} q_0 \xrightarrow{a} \textcircled{\scriptstyle q_1}

      Inductive Constructions

      Union R1R2R_1 | R_2:

      • New start state qnewq_{\text{new}}
      • ε\varepsilon-transitions to starts of M1M_1 and M2M_2
      • Accepting: F1F2F_1 \cup F_2

      NFA construction for union

      Concatenation R1R2R_1 R_2:

      • ε\varepsilon-transitions from accept states of M1M_1 to start of M2M_2
      • Start: start of M1M_1
      • Accepting: F2F_2

      NFA construction for concatenation

      Star RR^*:

      • New state qnewq_{\text{new}} (start and accepting)
      • ε\varepsilon-transition to old start
      • ε\varepsilon-transitions from old accepting states back to qnewq_{\text{new}}

      NFA construction for star

      Part (b): NFA to Regex

      The State Elimination Method

      For automaton M=(Q,Σ,Δ,s,F)M = (Q, \Sigma, \Delta, s, F), compute regular expression for each “region” of the automaton.

      A region (q,S,q)(q, S, q') consists of:

      • Start state qq
      • End state qq'
      • Set SS of intermediate states allowed

      The regional language LM(q,S,q)L_M(q, S, q') contains strings taking qq to qq' through only states in SS.

      Recursive Definition

      Base (S=S = \emptyset):

      LM(q,,q)={{ε}{a:qaq}q=q{a:qaq}qqL_M(q, \emptyset, q') = \begin{cases} \{\varepsilon\} \cup \{a : q \xrightarrow{a} q'\} & q = q' \\ \{a : q \xrightarrow{a} q'\} & q \neq q' \end{cases}

      This is a finite language, hence regular (expressible as finite union of regex).

      Inductive step: Pick q0Sq_0 \in S. Define S=S{q0}S' = S \setminus \{q_0\}.

      LM(q,S,q)=LM(q,S,q)LM(q,S,q0)LM(q0,S,q0)LM(q0,S,q)L_M(q, S, q') = L_M(q, S', q') \cup L_M(q, S', q_0) \cdot L_M(q_0, S', q_0)^* \cdot L_M(q_0, S', q')

      Intuition

      Paths through SS either:

      • Don’t visit q0q_0 at all: LM(q,S,q)L_M(q, S', q')
      • Visit q0q_0 at least once: go to q0q_0, loop zero or more times, go to qq'

      Final Expression

      The language of MM is:

      L(M)=qfFLM(s,Q,qf)L(M) = \bigcup_{q_f \in F} L_M(s, Q, q_f)

      where ss is the start state and QQ is all states.

      Example

      Simple Automaton

      Simple NFA for regex conversion

      q0aq1q_0 \xrightarrow{a} q_1, q1bq0q_1 \xrightarrow{b} q_0, q1cq1q_1 \xrightarrow{c} q_1

      Start: q0q_0, Accepting: {q1}\{q_1\}

      Computation

      LM(q0,{q0,q1},q1)L_M(q_0, \{q_0, q_1\}, q_1):

      • Either go directly q0q1q_0 \to q_1 on aa: contributes aa
      • Or go to q0q_0, loop, and exit: contributes q0aq1cq1bq0aq1q_0 \xrightarrow{a} q_1 \xrightarrow{c} q_1 \xrightarrow{b} q_0 \xrightarrow{a} q_1 pattern

      Result (simplified): a(cba)ca(c|ba)^*c^*| or via careful analysis: language of strings with more aa‘s than bb‘s modulo some pattern.


      Summary

      • Kleene’s theorem: regular expressions \Leftrightarrow finite automata
      • Regex to NFA: inductive construction using ε\varepsilon-transitions
      • NFA to regex: region decomposition, eliminate states
      • Regular = automata-recognisable = regex-definable
    • Compiling Regular Expressions to NFA

      Overview

      We construct an NFA-ε\varepsilon for each regular expression by induction on the structure of the regex.

      Each construction maintains the invariant that:

      • The NFA has exactly one start state (no incoming ε\varepsilon-transitions)
      • The NFA has exactly one accepting state (no outgoing ε\varepsilon-transitions)

      This modular approach enables clean composition.

      Base Cases

      Epsilon: ε\varepsilon

      NFA with one state that is both start and accepting.

      NFA for epsilon

      q\xrightarrow{\quad} \textcircled{\scriptstyle q}

      L(Mε)={ε}L(M_\varepsilon) = \{\varepsilon\}

      Empty: \emptyset

      NFA with one state, start but not accepting.

      NFA for empty set

      q\xrightarrow{\quad} q

      L(M)=L(M_\emptyset) = \emptyset

      Symbol: aa

      NFA with two states with a single aa-labelled transition.

      NFA for symbol a

      q0aq1\xrightarrow{\quad} q_0 \xrightarrow{a} \textcircled{\scriptstyle q_1}

      L(Ma)={a}L(M_a) = \{a\}

      Union: R1R2R_1 | R_2

      Construction

      1. Create new start state qsq_s
      2. Create new accepting state qfq_f
      3. Add ε\varepsilon-transitions:
        • qsεstart(M1)q_s \xrightarrow{\varepsilon} \text{start}(M_1)
        • qsεstart(M2)q_s \xrightarrow{\varepsilon} \text{start}(M_2)
        • accept(M1)εqf\text{accept}(M_1) \xrightarrow{\varepsilon} q_f
        • accept(M2)εqf\text{accept}(M_2) \xrightarrow{\varepsilon} q_f

      NFA construction for union

      Properties

      • Number of states: M1+M2+2|M_1| + |M_2| + 2
      • Linear in regex size

      Concatenation: R1R2R_1 R_2

      Construction

      1. Add ε\varepsilon-transition from accept(M1)\text{accept}(M_1) to start(M2)\text{start}(M_2)
      2. Start state: start(M1)\text{start}(M_1)
      3. Accepting state: accept(M2)\text{accept}(M_2)

      NFA construction for concatenation

      Properties

      • Number of states: M1+M2|M_1| + |M_2|
      • No new states added

      Kleene Star: RR^*

      Construction

      1. Create new start/accepting state q0q_0
      2. Add ε\varepsilon-transition q0εstart(M)q_0 \xrightarrow{\varepsilon} \text{start}(M)
      3. Add ε\varepsilon-transition from accept(M)\text{accept}(M) to start(M)\text{start}(M) (loop)
      4. Add ε\varepsilon-transition from accept(M)\text{accept}(M) to q0q_0
      5. q0q_0 is the new accepting state

      NFA construction for Kleene star

      Alternative Construction

      Some presentations use:

      • q0q_0 is start, εstart(M)\varepsilon \to \text{start}(M)
      • Accepting states include accept(M)\text{accept}(M) and a new state
      • Loop back from accept(M)\text{accept}(M) to start(M)\text{start}(M)

      Properties

      • Number of states: M+1|M| + 1
      • Accepts ε\varepsilon and any concatenation of strings from L(M)L(M)

      Size Analysis

      Number of States

      For regex RR of size nn (number of symbols and operators):

      MR=O(n)|M_R| = O(n)

      The Thompson construction builds NFAs with at most 2n2n states.

      Comparison to DFA

      The equivalent DFA may have up to 2MR2^{|M_R|} states.

      The exponential blowup is unavoidable in the worst case.

      Correctness

      Theorem

      For any regex RR, the construction produces NFA-ε\varepsilon MRM_R with L(MR)=L(R)L(M_R) = L(R).

      Proof

      By structural induction on RR.

      Base cases: Direct verification for ε\varepsilon, \emptyset, and symbols.

      Union: wL(MR1R2)w \in L(M_{R_1|R_2}) iff there is a path through M1M_1 or M2M_2. This happens iff wL(M1)w \in L(M_1) or wL(M2)w \in L(M_2), iff wL(R1)L(R2)=L(R1R2)w \in L(R_1) \cup L(R_2) = L(R_1 | R_2).

      Concatenation: wL(MR1R2)w \in L(M_{R_1 R_2}) iff we can split w=uvw = uv with path through M1M_1 on uu then through M2M_2 on vv. This happens iff uL(R1)u \in L(R_1) and vL(R2)v \in L(R_2), iff wL(R1)L(R2)w \in L(R_1) \cdot L(R_2).

      Star: Similar analysis using the fact that the construction accepts ε\varepsilon and any concatenation of strings from L(M)L(M).


      Summary

      • Thompson’s construction: linear-size NFA for any regex
      • Union: new start with ε\varepsilon-edges to both branches
      • Concatenation: ε\varepsilon-edge from accept of first to start of second
      • Star: new start/accept with loop back
      • Total states: O(R)O(|R|) where R|R| is regex size
    • Decompiling NFA to Regular Expressions

      Goal

      Given an NFA MM, construct a regular expression RR with L(R)=L(M)L(R) = L(M).

      The State Elimination Method

      Setup

      Given NFA M=(Q,Σ,Δ,s,F)M = (Q, \Sigma, \Delta, s, F), we compute a regex for paths through MM.

      Normalisation

      First, transform MM to have:

      1. A single start state (with no incoming edges)
      2. A single accepting state (with no outgoing edges)

      Add new start snews_{\text{new}} with ε\varepsilon-edge to ss.

      Add new accept fnewf_{\text{new}} with ε\varepsilon-edges from all states in FF.

      Regional Languages

      Definition

      A region (p,S,q)(p, S, q) specifies:

      • Source state pp
      • Target state qq
      • Set SS of states allowed as intermediate

      The regional language R(p,S,q)R(p, S, q) is the set of strings labelling paths from pp to qq using only states in SS as intermediate nodes.

      Base Case: S=S = \emptyset

      R(p,,q)={εa1akp=q and paipa1akpaiqno direct pathR(p, \emptyset, q) = \begin{cases} \varepsilon \cup a_1 | \cdots | a_k & p = q \text{ and } p \xrightarrow{a_i} p \\ a_1 | \cdots | a_k & p \xrightarrow{a_i} q \\ \emptyset & \text{no direct path} \end{cases}

      Where a1,,aka_1, \ldots, a_k label the direct edges from pp to qq.

      Inductive Step

      For S=S{r}S = S' \cup \{r\} where rSr \notin S':

      R(p,S,q)=R(p,S,q)R(p,S,r)R(r,S,r)R(r,S,q)R(p, S, q) = R(p, S', q) \cup R(p, S', r) \cdot R(r, S', r)^* \cdot R(r, S', q)

      Interpretation

      Paths from pp to qq through SS either:

      1. Don’t visit rr at all (stay in SS'): R(p,S,q)R(p, S', q)
      2. Visit rr at least once:
        • Go prp \to r staying in SS': R(p,S,r)R(p, S', r)
        • Loop at rr zero or more times: R(r,S,r)R(r, S', r)^*
        • Go rqr \to q staying in SS': R,S,q)R, S', q)

      Algorithm

      1. Normalise: single start ss, single accept ff
      2. Compute R(s,Q{s,f},f)R(s, Q \setminus \{s, f\}, f) by eliminating states one by one
      3. The result is L(M)L(M)

      Order of Elimination

      The order in which states are eliminated does not affect correctness, but can affect the size of the resulting regex.

      Common strategies:

      • Eliminate states with few transitions first
      • Eliminate “middle” states before endpoints

      Example

      Automaton

      Two-state NFA: q0aq1bq0q_0 \xrightarrow{a} q_1 \xrightarrow{b} q_0, with q1q_1 accepting.

      Two state NFA example

      Computation

      We want R(q0,{q0,q1},q1)R(q_0, \{q_0, q_1\}, q_1).

      Let S={q0,q1}S = \{q_0, q_1\}.

      Eliminate in some order. Suppose we eliminate q0q_0 first.

      Actually, with normalisation, we have start ss and accept ff. The internal states are {q0,q1}\{q_0, q_1\}.

      Compute R(s,{q0,q1},f)R(s, \{q_0, q_1\}, f):

      • Paths: sεq0aq1εfs \xrightarrow{\varepsilon} q_0 \xrightarrow{a} q_1 \xrightarrow{\varepsilon} f, or
      • q0aq1bq0aq1εfq_0 \xrightarrow{a} q_1 \xrightarrow{b} q_0 \xrightarrow{a} q_1 \xrightarrow{\varepsilon} f, etc.

      After elimination: the language is a(ba)a(ba)^* plus accounting for ε\varepsilon-transitions.

      Size Considerations

      Worst Case

      The regex may grow exponentially as states are eliminated due to the union and concatenation operations.

      Optimisation

      • Factor common subexpressions
      • Use algebraic simplifications (e.g., R=\emptyset \cdot R = \emptyset, εR=R\varepsilon \cdot R = R)
      • Choose good elimination order

      Closure under Complement

      Using automaton-to-regex conversion:

      1. Convert regex to DFA (via NFA)
      2. Swap accepting and non-accepting states
      3. Convert back to regex

      This shows regular languages are closed under complement.


      Summary

      • Compute regex R(p,S,q)R(p, S, q) for paths through region
      • Base case: direct transitions
      • Inductive step: R(p,S,q)=R(p,S,q)R(p,S,r)R(r,S,r)R(r,S,q)R(p, S, q) = R(p, S', q) \cup R(p, S', r) \cdot R(r, S', r)^* \cdot R(r, S', q)
      • Eliminate all internal states to get the full language
    • Closure Properties of Regular Languages

      Overview

      Regular languages are closed under many operations, meaning: applying the operation to regular languages yields another regular language.

      This is proved using automata or regular expressions.

      Union

      Theorem: If L1L_1 and L2L_2 are regular, then L1L2L_1 \cup L_2 is regular.

      Proof (NFA)

      Given DFAs M1M_1 and M2M_2:

      1. Create new start state s0s_0
      2. Add ε\varepsilon-transitions to s1s_1 and s2s_2
      3. Accepting states: F1F2F_1 \cup F_2

      Proof (Regex)

      If Li=L(Ri)L_i = L(R_i), then L1L2=L(R1R2)L_1 \cup L_2 = L(R_1 | R_2).

      Concatenation

      Theorem: If L1L_1 and L2L_2 are regular, then L1L2L_1 \cdot L_2 is regular.

      Proof (NFA)

      Given NFAs M1,M2M_1, M_2:

      1. Add ε\varepsilon-transitions from F1F_1 to s2s_2
      2. Start: s1s_1
      3. Accepting: F2F_2

      Proof (Regex)

      If Li=L(Ri)L_i = L(R_i), then L1L2=L(R1R2)L_1 \cdot L_2 = L(R_1 R_2).

      Kleene Star

      Theorem: If LL is regular, then LL^* is regular.

      Proof (NFA)

      1. Create new start/accept state q0q_0
      2. Add ε\varepsilon-transition q0sq_0 \to s
      3. Add ε\varepsilon-transitions from each fFf \in F to ss
      4. Add ε\varepsilon-transition from each fFf \in F to q0q_0
      5. Accepting: {q0}\{q_0\}

      Proof (Regex)

      If L=L(R)L = L(R), then L=L(R)L^* = L(R^*).

      Complement

      Theorem: If LL is regular, then Lˉ=ΣL\bar{L} = \Sigma^* \setminus L is regular.

      Proof (DFA)

      Given DFA M=(Q,Σ,δ,s,F)M = (Q, \Sigma, \delta, s, F):

      Define Mˉ=(Q,Σ,δ,s,QF)\bar{M} = (Q, \Sigma, \delta, s, Q \setminus F).

      Swap accepting and non-accepting states.

      Note: Requires complete DFA (all transitions defined).

      Intersection

      Theorem: If L1L_1 and L2L_2 are regular, then L1L2L_1 \cap L_2 is regular.

      Proof (DFA Product Construction)

      Given DFAs M1=(Q1,Σ,δ1,s1,F1)M_1 = (Q_1, \Sigma, \delta_1, s_1, F_1) and M2=(Q2,Σ,δ2,s2,F2)M_2 = (Q_2, \Sigma, \delta_2, s_2, F_2):

      Define product DFA M=(Q,Σ,δ,s,F)M = (Q, \Sigma, \delta, s, F):

      • Q=Q1×Q2Q = Q_1 \times Q_2
      • δ((q1,q2),a)=(δ1(q1,a),δ2(q2,a))\delta((q_1, q_2), a) = (\delta_1(q_1, a), \delta_2(q_2, a))
      • s=(s1,s2)s = (s_1, s_2)
      • F=F1×F2F = F_1 \times F_2

      Product accepts iff both components are accepting.

      Proof (De Morgan)

      L1L2=Lˉ1Lˉ2L_1 \cap L_2 = \overline{\bar{L}_1 \cup \bar{L}_2}

      Use closure under complement and union.

      Difference

      Theorem: If L1L_1 and L2L_2 are regular, then L1L2L_1 \setminus L_2 is regular.

      Proof

      L1L2=L1Lˉ2L_1 \setminus L_2 = L_1 \cap \bar{L}_2

      Use intersection and complement.

      Homomorphism

      Theorem: If LL is regular and h:ΣΓh : \Sigma^* \to \Gamma^* is a homomorphism, then h(L)h(L) is regular.

      Proof (Regex)

      Apply hh to each symbol in the regex.

      Inverse Homomorphism

      Theorem: If LΓL \subseteq \Gamma^* is regular and h:ΣΓh : \Sigma^* \to \Gamma^* is a homomorphism, then h1(L)h^{-1}(L) is regular.

      Proof (DFA)

      Given DFA MM for LL, construct MM' where:

      δ(q,a)=δ^(q,h(a))\delta'(q, a) = \hat{\delta}(q, h(a))

      Run MM on the image of each input symbol.

      Reversal

      Theorem: If LL is regular, then LR={wR:wL}L^R = \{w^R : w \in L\} is regular.

      Proof

      Reverse all transitions in an NFA, swap start and accept, convert to DFA.

      Properties Summary

      OperationConstructionNotes
      UnionNFA with ε\varepsilon-edgesEasy
      ConcatenationNFA with ε\varepsilon-edgesEasy
      StarNFA with loop backEasy
      ComplementSwap accept/non-accept in DFANeeds complete DFA
      IntersectionProduct DFAQuadratic states
      DifferenceL1Lˉ2L_1 \cap \bar{L}_2Via intersection and complement
      ReversalReverse NFASwap start/accept
      HomomorphismApply to regexSymbol-wise
      Inverse homomorphismDFA transformationEfficient

      Summary

      • Regular languages are closed under: \cup, \cdot, *, ˉ\bar{\cdot}, \cap, \setminus, reversal
      • Union/concat/star: easy with NFAs
      • Complement: easy with DFAs, hard with NFAs
      • Intersection: product construction or complement + union
    • The Pumping Lemma

      Statement

      Theorem (Pumping Lemma for Regular Languages):

      Let LL be a regular language. There exists p1p \geq 1 (the pumping length) such that for every wLw \in L with wp|w| \geq p:

      ww can be written as w=xyzw = xyz where:

      1. xyp|xy| \leq p
      2. y1|y| \geq 1 (so yεy \neq \varepsilon)
      3. xyizLxy^iz \in L for all i0i \geq 0

      Intuition

      If a string is long enough and the language is regular, then some part of the string can be “pumped” (repeated) any number of times while staying in the language.

      The pumped part corresponds to a loop in an accepting computation of the DFA.

      Proof Idea

      Let M=(Q,Σ,δ,s,F)M = (Q, \Sigma, \delta, s, F) be a DFA for LL with Q=n|Q| = n.

      Set p=np = n.

      For any wLw \in L with wn|w| \geq n: the computation visits n+1n+1 states in sequence. By the pigeonhole principle, some state is visited twice.

      The part of ww that causes the loop can be pumped.

      Formal Argument

      Path through DFA: s=q0q1qw=qfs = q_0 \to q_1 \to \cdots \to q_{|w|} = q_f.

      Since wn+1|w| \geq n+1 states are visited, but there are only nn states, some qi=qjq_i = q_j for i<ji < j.

      Let x=w[0..i]x = w[0..i] (before the loop), y=w[i..j]y = w[i..j] (the loop), z=w[j..w]z = w[j..|w|] (after).

      • Condition 1: Loop occurs within first nn states, so xyn=p|xy| \leq n = p
      • Condition 2: i<ji < j, so y1|y| \geq 1
      • Condition 3: Repeating or omitting the loop keeps the computation in the accepting state

      Using the Pumping Lemma

      To Prove Non-regularity

      1. Assume LL is regular with pumping length pp
      2. Choose a string wLw \in L with wp|w| \geq p
      3. Show that for ALL possible decompositions w=xyzw = xyz satisfying conditions 1 and 2:
        • There exists ii such that xyizLxy^iz \notin L
      4. Conclude: contradiction, so LL is not regular

      Key Point

      You must show that the pumping lemma fails for every possible choice of x,y,zx, y, z satisfying the conditions.

      Application: L={anbn:n0}L = \{a^n b^n : n \geq 0\}

      Proof

      Assume LL is regular with pumping length pp.

      Choose w=apbpw = a^p b^p (in LL, length p\geq p).

      For any decomposition w=xyzw = xyz with xyp|xy| \leq p and y1|y| \geq 1:

      • Since xyp|xy| \leq p, the substring xyxy consists of only aa‘s
      • Thus y=aky = a^k for some k1k \geq 1

      Consider xy2z=ap+kbpxy^2z = a^{p+k} b^p:

      • Number of aa‘s is p+k>pp+k > p
      • Number of bb‘s is pp
      • Not equal, so xy2zLxy^2z \notin L

      Contradiction. Thus LL is not regular.

      Application: L={ww:w{a,b}}L = \{ww : w \in \{a, b\}^*\}

      Proof

      Assume LL is regular with pumping length pp.

      Choose w=apbapbw = a^p b a^p b (in LL, length 2p+2p2p+2 \geq p).

      For any decomposition w=xyzw = xyz with xyp|xy| \leq p and y1|y| \geq 1:

      • xyp|xy| \leq p means xyxy lies within the first apba^p b (actually, within first apa^p)
      • So y=aky = a^k for some k1k \geq 1

      Consider xy2z=ap+kbapbxy^2z = a^{p+k} b a^p b:

      • This is not of the form wwww (different lengths of prefix)
      • Alternatively: after pumping, the string is ap+kbapba^{p+k} b a^p b, not a square

      Contradiction.

      Application: L={an2:n0}L = \{a^{n^2} : n \geq 0\}

      Proof

      Choose w=ap2w = a^{p^2}.

      The pumped string is ap2+ka^{p^2+k} for some kk with 1kp1 \leq k \leq p.

      We need p2+kp^2 + k to be a perfect square.

      But (p+1)2=p2+2p+1>p2+k(p+1)^2 = p^2 + 2p + 1 > p^2 + k for kpk \leq p.

      So ap2+kLa^{p^2+k} \notin L.

      Limitations of the Pumping Lemma

      The pumping lemma is a necessary condition for regularity, not sufficient.

      Some non-regular languages satisfy the pumping lemma conditions.

      For proving non-regularity, use:

      • Pumping lemma
      • Myhill-Nerode theorem
      • Closure properties

      Summary

      • Pumping lemma: long strings in regular languages have pumpable parts
      • Proof uses pigeonhole principle on DFA states
      • Use: assume regular, find a counterexample string, show pumping fails
      • Applications: anbna^n b^n, wwww, {an2}\{a^{n^2}\} are not regular
      • Not a characterisation: necessary, not sufficient
  • Other Resources

    • Examinable Proofs Reference

      This document consolidates all proofs you should be able to reproduce under exam conditions, organised by topic.


      Part I: Proofs

      1. Irrationality of 2\sqrt{2}

      Theorem: 2\sqrt{2} is irrational.

      Proof:

      Suppose for contradiction 2=pq\sqrt{2} = \frac{p}{q} where gcd(p,q)=1\gcd(p, q) = 1.

      Squaring: 2=p2q22 = \frac{p^2}{q^2}, so p2=2q2p^2 = 2q^2.

      Thus p2p^2 is even, so pp is even. Write p=2kp = 2k.

      Substituting: 4k2=2q24k^2 = 2q^2, so q2=2k2q^2 = 2k^2.

      Thus q2q^2 is even, so qq is even.

      But pp and qq both even contradicts gcd(p,q)=1\gcd(p, q) = 1. \square


      2. Infinitude of Primes (Euclid)

      Theorem: There are infinitely many primes.

      Proof:

      Suppose finitely many primes: p1,,pnp_1, \ldots, p_n.

      Let N=p1pn+1N = p_1 \cdots p_n + 1.

      Since N>1N > 1, NN has a prime divisor pp.

      If p=pip = p_i for some ii, then pp1pnp \mid p_1 \cdots p_n and pNp \mid N, so p(Np1pn)=1p \mid (N - p_1 \cdots p_n) = 1.

      No prime divides 1. Contradiction.

      Thus pp is not in our list. \square


      3. Fermat’s Little Theorem

      Theorem: If pp is prime and pap \nmid a, then ap11(modp)a^{p-1} \equiv 1 \pmod{p}.

      Proof:

      Consider a,2a,,(p1)a(modp)a, 2a, \ldots, (p-1)a \pmod{p}.

      Non-zeroness: Each ia≢0(modp)ia \not\equiv 0 \pmod{p} since pap \nmid a and pip \nmid i.

      Distinctness: If iaja(modp)ia \equiv ja \pmod{p}, then p(ij)ap \mid (i-j)a. By Euclid’s lemma, p(ij)p \mid (i-j). But ij<p|i-j| < p, so i=ji = j.

      Thus {a,2a,,(p1)a}\{a, 2a, \ldots, (p-1)a\} = {1,2,,p1}(modp)\{1, 2, \ldots, p-1\} \pmod{p}.

      Taking products: ap1(p1)!(p1)!(modp)a^{p-1}(p-1)! \equiv (p-1)! \pmod{p}.

      Since gcd((p1)!,p)=1\gcd((p-1)!, p) = 1, cancel: ap11(modp)a^{p-1} \equiv 1 \pmod{p}. \square


      Part II: Sets, Relations, Functions

      4. Relational Composition is Associative

      Theorem: (TS)R=T(SR)(T \circ S) \circ R = T \circ (S \circ R).

      Proof:

      a((TS)R)d    c.aRcc(TS)da((T \circ S) \circ R)d \iff \exists c. aRc \land c(T \circ S)d     c,b.aRccSbbTd\iff \exists c, b. aRc \land cSb \land bTd

      a(T(SR))d    b.a(SR)bbTda(T \circ (S \circ R))d \iff \exists b. a(S \circ R)b \land bTd     b,c.aRccSbbTd\iff \exists b, c. aRc \land cSb \land bTd

      Both equivalent to b,c.aRccSbbTd\exists b, c. aRc \land cSb \land bTd. \square


      5. Identity Laws for Composition

      Theorem: idBR=R=RidA\text{id}_B \circ R = R = R \circ \text{id}_A.

      Proof:

      a(idBR)b    c.aRccidBb    c.aRcc=b    aRba(\text{id}_B \circ R)b \iff \exists c. aRc \land c\,\text{id}_B\,b \iff \exists c. aRc \land c = b \iff aRb.

      a(RidA)b    c.aidAccRb    c.a=ccRb    aRba(R \circ \text{id}_A)b \iff \exists c. a\,\text{id}_A\,c \land cRb \iff \exists c. a = c \land cRb \iff aRb. \square


      6. Reflexive-Transitive Closure is Smallest Preorder

      Theorem: RR^{\circ *} is the smallest preorder containing RR.

      Proof:

      RR^{\circ *} is a preorder: reflexive (contains id\text{id}), transitive (path concatenation).

      RRR \subseteq R^{\circ *}: Clear since R1=RR^{\circ 1} = R.

      Smallest: Let PP be a preorder with RPR \subseteq P. By induction on nn: RnPR^{\circ n} \subseteq P.

      • Base: idP\text{id} \subseteq P by reflexivity.
      • Step: R(n+1)=RRnPPPR^{\circ(n+1)} = R \circ R^{\circ n} \subseteq P \circ P \subseteq P by transitivity.

      Thus R=nRnPR^{\circ *} = \bigcup_n R^{\circ n} \subseteq P. \square


      7. Equivalence Relations \leftrightarrow Partitions Bijection

      Theorem: Φ:EqRel(A)Part(A)\Phi : \text{EqRel}(A) \to \text{Part}(A) by Φ(E)={[a]E:aA}\Phi(E) = \{[a]_E : a \in A\} is a bijection.

      Proof:

      Well-defined: Equivalence classes partition AA (non-empty by reflexivity, disjoint by transitivity/symmetry, cover by definition).

      Inverse: Given partition PP, define E={(a,b):BP.a,bB}E = \{(a, b) : \exists B \in P. a, b \in B\}.

      This is an equivalence relation (reflexive: aa in unique block; symmetric: trivial; transitive: blocks disjoint).

      Inverses: Check Φ1(Φ(E))=E\Phi^{-1}(\Phi(E)) = E (elements in same class iff equivalent) and Φ(Φ1(P))=P\Phi(\Phi^{-1}(P)) = P (blocks are the equivalence classes). \square


      8. Isomorphisms are Bijective

      Theorem: f:ABf : A \to B is an isomorphism iff ff is bijective.

      Proof:

      ()(\Rightarrow): If f1f^{-1} exists: injective (if f(a1)=f(a2)f(a_1) = f(a_2), then a1=f1(f(a1))=f1(f(a2))=a2a_1 = f^{-1}(f(a_1)) = f^{-1}(f(a_2)) = a_2); surjective (for bb, b=f(f1(b))b = f(f^{-1}(b))).

      ()(\Leftarrow): Define f1(b)=f^{-1}(b) = the unique aa with f(a)=bf(a) = b. Verify f1f=idf^{-1} \circ f = \text{id} and ff1=idf \circ f^{-1} = \text{id}. \square


      9. Sections and Idempotents

      Theorem: (s,r)(s, r) section-retraction pair \leftrightarrow e=sre = s \circ r idempotent.

      Proof:

      ()(\Rightarrow): ee=srsr=sidr=sr=ee \circ e = s \circ r \circ s \circ r = s \circ \text{id} \circ r = s \circ r = e.

      ()(\Leftarrow): Given idempotent ee, let B=Im(e)B = \text{Im}(e). Define r(a)=e(a)r(a) = e(a), s(b)=bs(b) = b (inclusion). Then r(s(b))=e(b)=br(s(b)) = e(b) = b since bIm(e)b \in \text{Im}(e) means b=e(a)b = e(a) for some aa. \square


      10. Cantor’s Theorem

      Theorem: No surjection AP(A)A \twoheadrightarrow \mathcal{P}(A) exists.

      Proof:

      Suppose e:AP(A)e : A \twoheadrightarrow \mathcal{P}(A). Define U={xA:xe(x)}U = \{x \in A : x \notin e(x)\}.

      Since ee surjective, a.e(a)=U\exists a. e(a) = U.

      Is aUa \in U?

      • If yes: aUae(a)=Ua \in U \Rightarrow a \notin e(a) = U. Contradiction.
      • If no: aUae(a)=Ua \notin U \Rightarrow a \in e(a) = U. Contradiction.

      Thus no such ee exists. \square


      11. Well-Founded Induction

      Theorem: If (yx.P(y))P(x)(\forall y \prec x. P(y)) \Rightarrow P(x) for all xx, then x.P(x)\forall x. P(x).

      Proof:

      Suppose S={x:¬P(x)}S = \{x : \neg P(x)\} \neq \emptyset.

      By well-foundedness, SS has minimal element mm.

      For all ymy \prec m: ySy \notin S, so P(y)P(y) holds.

      By assumption, P(m)P(m) holds, contradicting mSm \in S.

      Thus S=S = \emptyset. \square


      Part III: Formal Languages and Automata

      12. Rule Induction

      Theorem: If SS is closed under all rules in RR, then XRSX_R \subseteq S.

      Proof:

      Induction on derivation height.

      Base: Axioms have height 1. Since SS closed under rules with no premises, axioms are in SS.

      Step: Derivation of height n+1n+1 uses rule u1,,ukv\frac{u_1, \ldots, u_k}{v} where each uiu_i derived by height n\leq n.

      By IH, each uiSu_i \in S. Since SS closed under the rule, vSv \in S. \square


      13. Subset Construction Correctness

      Theorem: L(P(M))=L(M)L(\mathcal{P}(M)) = L(M).

      Proof:

      Soundness (L(P(M))L(M)L(\mathcal{P}(M)) \subseteq L(M)): DFA path sSnFs' \to S_n \cap F \neq \emptyset. Build NFA path by tracing elements in each SiS_i back to NFA states. Some qfSnFq_f \in S_n \cap F reachable from ss.

      Completeness (L(M)L(P(M))L(M) \subseteq L(\mathcal{P}(M))): NFA path swqfs \Rightarrow_w q_f. Track reachable states SiS_i at each step. qfSnq_f \in S_n, so SnFS_n \cap F \neq \emptyset. \square


      14. Kleene’s Theorem

      Theorem: LL is regular iff L=L(R)L = L(R) for some regex RR.

      Proof:

      Regex \Rightarrow NFA: Inductive construction (Thompson) for ε\varepsilon, \emptyset, symbols, union, concat, star. Each preserves language.

      NFA \Rightarrow Regex: Region decomposition. Compute R(q,S,q)R(q, S, q') for paths through regions. Eliminate all states: L(M)=qfFR(s,Q,qf)L(M) = \bigcup_{q_f \in F} R(s, Q, q_f). \square


      15. Pumping Lemma

      Theorem: For LL regular, p.wL\exists p. \forall w \in L with wp|w| \geq p: w=xyzw = xyz with xyp|xy| \leq p, y1|y| \geq 1, and i.xyizL\forall i. xy^iz \in L.

      Proof:

      Let DFA have nn states. Set p=np = n.

      For wn|w| \geq n: computation visits n+1n+1 states. By pigeonhole, some state repeated.

      Let xx = prefix before loop, yy = loop, zz = suffix.

      • xyn=p|xy| \leq n = p (loop within first nn states)
      • y1|y| \geq 1 (loop non-trivial)
      • Can repeat or omit loop; stays accepting. \square

      Summary

      These 15 proofs form the core of examinable material. Key strategies:

      1. Contradiction: Assume negation, derive 2\sqrt{2}, primes, Cantor
      2. Induction: Natural numbers, derivation height, state elimination
      3. Construction: Explicit bijections, automata constructions
      4. Unfolding definitions: Relational composition, closure properties