Modern Definition of Probability. Set Theoretic Approach

Sample Space » Sample Points

Sample Space » All Possible Elementary Events

The sample space of a random experiment is the set of all possible elementary events or outcomes for the experiment.
  • It is also called the universal sample space.
  • The sample space is a nonempty set.
  • It is denoted as - S, O or U [for "universe"].

Examples

  1. Experiment : Throwing a coin

    Sample Space : S = {H, T}

    1. "S" represents the sample space.
    2. "H" the elementary event of getting a head
    3. "T" the elementary event of getting a tail.

  2. Experiment : Rolling a dice

    Sample Space : O = {1, 2, 3, 4, 5, 6}

    1. "O" represents the sample space,
    2. "1" the elementary event of the number 1 appearing on the top of the dice,
    3. "2" the elementary event of the number 1 appearing on the top of the dice,
    4. ...
    5. ...
    6. ...
    7. "6" the elementary event of the number 6 appearing on the top of the dice.

  3. Experiment : Tossing 3 coins

    Sample Space : S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}

    1. "S" represents the sample space.
    2. "HHT" the elementary event of the 1st and 2nd coins showing up heads and the 3rd coin showing up tails,
    3. ...
    4. "TTT" the elementary event of all three coins showing up tails.

  4. Experiment : Rolling 2 dice

    S = { (1,1), (1,2), .., .., .., (1,6)
    (2,1), (2,2), .., .., .., (2,6)
    ...
    ...
    (6,1), (6,2), .., .., .., (6,6) }

  5. Experiment : Throwing 3 dice

    S = {(1,1,1), (1,1,2),..,.., .., (1,1,6)
    (1,2,1), (1,2,2),..,.., .., (1,2,6)
    ...
    ...
    (6,6,1), (6,6,2),..,.., .., (6,6,6)}

Events : Set notation form

Any subset of the sample space forms an an event in relation to the experiment.

Examples

  1. Experiment : Rolling a dice

    Sample space : O = {1, 2, 3, 4, 5, 6}

    1. A : Event of getting an even number, represented as A = {2, 4, 6}
    2. H : Event of getting a prime number, represented as H = {2, 3, 5}
    3. F: Event of getting a multiple of 3, represented as F = {3, 6}

    A, H and F are all Subsets of O

  2. Experiment : Tossing 3 coins

    Sample space : S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}

    1. C : Event of getting two heads, represented as C = {HHT, HTH, THH}
    2. G : Event of getting all three of the same kind, represented as G = {HHH, TTT}

    C and G are Subsets of S.

Sample Points » Elementary Events

Each elementary event within a sample space is called a sample point.

Experiment : Tossing 3 coins

Sample space : S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}

Each of the elementary events in the sample space form a sample point.

A subset of the sample space with just a single element (a sample point) forms an elementary event for the experiment.

The term sample point is used in two senses

  1. An element of a set.

    'HHH' representing the event of getting all heads is a sample point.

  2. A set with a single element.

    M = {'HHH'}, Set 'M' is a sample point.

Limitations of Classiscal/Mathematical/A-priori Definition of Probability

These limitations in the classical definition of probability led to the evolution of the modern definition of probability which is based on the concept of sets and has an axiomatic approach.
  • The Classical definition of probability limited its application only to situations where there are a finite number of possible outcomes. It mainly considered discrete events and its methods were mainly combinatorial.
  • The classical theory assumes finite outcomes in the experiment. This renders it inapplicable to some important random experiments, such as "tossing a coin until a head appears" which give rise to the possibility infinite set of outcomes.
  • Classical definition dealt with discrete variables. Analytical considerations compelled the incorporation of continuous variables into the theory wherein the classical definition was found lacking.
  • Another limitation of the classical definition was the condition that each possible outcome is 'equally likely". This rendered the definition circular - since probability is used to define the idea of probability.

The foundations of the Modern Probability theory were laid by Andrey Nikolaevich Kolmogorov who combined the notion of sample space (introduced by Richard von Mises) and the measure theory and presented his axiom system for probability theory 1933. This forms the axiomatic basis for modern probability theory.

Combinatorics

A branch of pure mathematics concerning the study of discrete (and usually finite) objects. Graph theory, Permutations, Combinations are some topics that fall within the purview of combinatorics.

Pure mathematics

"That part of mathematical activity that is done without explicit or immediate consideration of direct application". The pure status may not be permanent as what is "pure" in one era often becomes applied later.

Applied mathematics

A branch of mathematics that concerns itself with the application of mathematical knowledge to other domains.

Modern Definition of Probability : Axiomatic Approach

Each element x ∈ Ω has a related probability value attached to it such that it satisfies the following properties.

i f(x) ∈ [0, 1] for all x ∈ Ω

For every value of x in the sample space Ω, the probability function f(x) lies between zero & one.

The probability of the null event is 0.

ii.
Σ
x ∈ Ω
= P(Ei)
The Probability of an Event (E), (a subset of the sample space (Ω)) is given by P(E) = Σ
x ∈ E

The probability of an event is the sum of the probabilities of the individual elementary events forming the event.

The probability of the entire sample space is 1.

Modern Definition of Probability : Set Theoretic Approach

Probability of occurrence of an event is given by the ratio of the cardinality of the set of all the elementary events favorable/favourable to the occurrence of the event and the set of all possible elementary events in relation to the experiment i.e. the sample space.

Where

  • S : Sample space [i.e. the set of all possible elementary events in relation to the experiment]
  • A : Subset of S,
The probability of occurrence of an event "A"
=
Number of elements in set "A"
Number of elements in set "S"
(Or)
Cardinality of set "A"
Cardinality of set "S"
⇒ P(A) =
n(A)
n(S)

Probability of occurrence of Sample Space

In conducting an experiment, any one of the possible elementary events would occur.

⇒ Occurrence of an elementary event from the sample space is a certainity.

Probability of a certain event is 1 ⇒ P(S) = 1.

Set Theoretic Approach

The probability of occurrence of event "S" =
Number of elements in set "S"
Number of elements in set "S"
⇒ P(S) =
n(S)
n(S)
= 1

Probability Axioms » Kolmogorov Axioms

Axiom

In mathematics, an axiom is any starting assumption from which other statements are logically derived. It can be a sentence, a proposition, a statement or a rule that enables the construction of a formal system.

Unlike theorems, axioms cannot be derived by principles of deduction, nor are they demonstrable by formal proofs - simply because they are starting assumptions - there is nothing else they logically follow from (otherwise they would be called theorems). In many contexts, "axiom," "postulate," and "assumption" are used interchangeably.

Probability Axioms » Kolmogorov Axioms

The modern definition of probability is based on the Kolmogorov axioms

First Axiom

The probability of an event is a non-negative real number i.e. P (E) ≥ 0 ∀ E ∈ O

Second Axiom

The probability that any one of the elementary events in the entire sample set will occur is 1
⇒ P (S) = 1

Third Axiom

Any countable sequence of pair wise disjoint events E1, E2, ... satisfies
P(E1 U E2 U ...) =
Σ
i
P(Ei)
... 171819 ...