116

How should I understand the difference or relationship between binomial and Bernoulli distribution?

user122358
  • 2,552
  • 5
  • 20
  • 32

4 Answers4

133

A Bernoulli random variable has two possible outcomes: $0$ or $1$. A binomial distribution is the sum of independent and identically distributed Bernoulli random variables.

So, for example, say I have a coin, and, when tossed, the probability it lands heads is $p$. So the probability that it lands tails is $1-p$ (there are no other possible outcomes for the coin toss). If the coin lands heads, you win one dollar. If the coin lands tails, you win nothing.

For a single coin toss, the probability you win one dollar is $p$. The random variable that represents your winnings after one coin toss is a Bernoulli random variable.

Now, if you toss the coin $5$ times, your winnings could be any whole number of dollars from zero dollars to five dollars, inclusive. The probability that you win five dollars is $p^5$, because each coin toss is independent of the others, and for each coin toss the probability of heads is $p$.

What is the probability that you win exactly three dollars in five tosses? That would require you to toss the coin five times, getting exactly three heads and two tails. This can be achieved with probability $\binom{5}{3} p^3 (1-p)^2$. And, in general, if there are $n$ Bernoulli trials, then the sum of those trials is binomially distributed with parameters $n$ and $p$.

Note that a binomial random variable with parameter $n = 1$ is equivalent to a Bernoulli random variable, i.e. there is only one trial.

heropup
  • 112,408
  • 12
  • 89
  • 159
  • 7
    How should I read $\binom{5}{3} p^3 (1-p)^2$ ? Please understand if I am asking you the very basic thing, I am new to it. – user122358 Jun 18 '14 at 06:52
  • 20
    Say we represent the outcome of the five coin tosses as a list, and three of those tosses must be heads. So we can have $$\begin{align*}&\{H,H,H,T,T\}, \\ &\{H,H,T,H,T\}, \\ &\{H,H,T,T,H\}, \\ &\{H,T,H,H,T\}, \\ &\{H,T,H,T,H\}, \\ &\{H,T,T,H,H\}, \\ &\{T,H,H,H,T\}, \\ &\{T,H,H,T,H\}, \\ &\{T,H,T,H,H\}, \\ &\{T,T,H,H,H\}. \end{align*}$$ As you can see, there are $\binom{5}{3} = 10$ ways to choose the positions of the three heads among the five tosses; and for each of these outcomes, the probability of observing three heads is $p^3$ and two tails is $(1-p)^2$. – heropup Jun 18 '14 at 07:07
  • 2
    I want to point out this answer doesn’t answer the specific question in the original post - that is, what is the difference between a Bernoulli *distribution* and a binomial *distribution*. Michael Hardy’s answer below addresses this specific question. – Just Some Old Man Jun 25 '19 at 17:20
  • This is simply the best explanation...thanks – Narendra Jun 26 '19 at 07:30
31

All Bernoulli distributions are binomial distributions, but most binomial distributions are not Bernoulli distributions.

If $$ X=\begin{cases} 1 & \text{with probability }p, \\ 0 & \text{with probability }1-p, \end{cases} $$ then the probability distribution of the random variable $X$ is a Bernoulli distribution.

If $X=X_1+\cdots+X_n$ and each of $X_1,\ldots,X_n$ has a Bernoulli distribution with the same value of $p$ and they are independent, then $X$ has a binomial distribution, and the possible values of $X$ are $\{0,1,2,3,\ldots,n\}$. If $n=1$ then that binomial distribution is a Bernoulli distribution.

Michael Hardy
  • 1
  • 30
  • 276
  • 565
  • One question, if $n=1$ then why is $X_{i}$ a `distribution` ? – Neel Basu Apr 10 '18 at 09:41
  • 9
    @NeelBasu : $X_i$ is not a distribution; rather $X_i$ _has_ a distribution. $X_i$ is a random variable. – Michael Hardy Apr 10 '18 at 15:58
  • @Michael Hardy: you said $ X_1, ..., X_n $ has a Bernoulli distribution with the same value of $ p $... Does each random variable of X need to have the same value of p to be identically distributed? – oceanus Jan 10 '20 at 14:53
  • 2
    @ChangheeKang : "Identically distributed" means having the same probability distribution. If the value of $p$ is altered, so is the probability distribution. – Michael Hardy Jan 10 '20 at 19:05
  • Since there's only two outcomes X=0 and X=1, is it always the case that a Bernoulli distribution is essentially two points, one of height p(X=0) and the other (necessarily) of height 1 - p(X=0) ? – N.S. Jan 20 '21 at 22:39
22

A Bernoulli distribution is a special case of binomial distribution. Specifically, when $n=1$ the binomial distribution becomes Bernoulli distribution.

anita choudhary
  • 221
  • 2
  • 2
2

A Bernoulli random variable $X$ is a random variable that satisfies $P(X=1)=p$, $P(X=0)=1-p$. A canonical example is a coin flip which has $p=1/2$. In fact, you can think of a Bernoulli random variable is just a weighted coin, which comes up $1$ with some probability and $0$ otherwise. A binomial random variable with parameters $n,p$ is what you get when you count the number of $1$'s (successes) that come up in a string of $n$ independent Bernoulli random variables, each with parameter $p$. Another way to say this is that a binomial random variable is the sum of independent and identically distributed Bernoulli random variables.

crf
  • 5,319
  • 4
  • 29
  • 53