Machine Learning & Signals Learning
2 Continuous Random Variable
2.1 Characterization
CDF: The definition of the CDF (Definition 1.2.1) is identical for discrete and continuous variables: the probability that the value of a random variable \(X\) is less than or equal to a given value \(x\),
\(\seteqnumber{0}{2.}{0}\)\begin{equation} F_{X}(x) = \Pr (X\le x),\ x\in \mathbb {R}. \end{equation}
Continuous PDF: For a continuous PDF we have
\(\seteqnumber{0}{2.}{1}\)\begin{equation} \Pr (X\le x) = \Pr (X < x) \end{equation}
Probability density function (PDF): Assume the CDF is differentiable for all values of \(x\). We can then define,
\(\seteqnumber{1}{2.3}{0}\)\begin{align} f_{X}(x) &= \frac {d F_{X}(x)}{d x }\\ F_{X}(x) &= \int _{-\infty }^{x}f_{X}(p)\,dp \end{align}
Probability over an interval: Based on the definition of the PDF/CDF,
\(\seteqnumber{0}{2.}{3}\)\begin{equation} \Pr (a<X\leqslant b) = \int _{a}^{b}f_X(x)\,dx = F_X(b)-F_X(a) \end{equation}
Properties of the PDF: Range of values and “sum” of values,
\(\seteqnumber{1}{2.5}{0}\)\begin{align} f_{X}(x) &\ge 0 \\ \int _{-\infty }^\infty f_{X}(x)\,dx &= 1 = F_X(\infty ) \end{align}
The PDF of a random variable does not directly represent a probability, whereas the CDF does represent a probability. For a continuous random variable it can even hold that \(f_{X}(x) > 1\).
Probability of a single value: Since there are infinitely many values,
\(\seteqnumber{0}{2.}{5}\)\begin{equation} \Pr (X=a) = 0\quad \forall a\in \mathbb {R}. \end{equation}
The PDF of a discrete variable can be written as a continuous function of the form
\(\seteqnumber{0}{2.}{6}\)\begin{equation} f_X(x) = \sum _{k}p_X[x_k]\delta (x-x_k). \end{equation}
This relates the piecewise-constant \(F_{X}(x)\) of a discrete random variable to its derivative, which is the PDF.
Expectation: The expectation of a continuous random variable is
\(\seteqnumber{0}{2.}{7}\)\begin{equation} \E [X] = \int _{-\infty }^{\infty } xf_X(x)\,dx \end{equation}
Expectation of a function of a random variable:
\(\seteqnumber{0}{2.}{8}\)\begin{equation} \E [g(X)] = \int _{-\infty }^{\infty } g(x)f_X(x)\,dx \end{equation}
Since the computation of the variance (Section 1.2.3) is based on the expectation formula, it is identical for discrete and continuous variables.
Mode: The value \(x_0\) for which the PDF attains its maximum value,
\(\seteqnumber{0}{2.}{9}\)\begin{equation} x_0 = \argmax _{x} f_X(x) \end{equation}
The definition of the median is identical for discrete and continuous variables.
2.2 Distributions
2.2.1 Exponential Distribution
PDF: For \(X\sim \mathrm {Exp}(\lambda )\) we have1
\(\seteqnumber{0}{2.}{10}\)\begin{equation} f_X(x) = \begin{cases} \lambda \exp (-\lambda x) & x\ge 0\\ 0 & \text {otherwise} \end {cases} \end{equation}
1 In the literature (and also in MATLAB) another definition appears:
\(\seteqnumber{0}{2.}{10}\)\begin{equation*} f_X(x) = \begin{cases} \dfrac {1}{\mu }\exp \left (-\dfrac {x}{\mu }\right ) & x\ge 0\\ 0 & \text {otherwise} \end {cases}, \end{equation*}
where \(\mu = \dfrac {1}{\lambda }\).
-
Example 2.1: Compute the expectation of \(X\), where \(X\sim \mathrm {Exp}(\lambda )\).
Variance: When \(X\sim \mathrm {Exp}(\lambda )\),
\(\seteqnumber{0}{2.}{12}\)\begin{equation} \Var [X] = \dfrac {1}{\lambda ^2} \end{equation}
-
Proof. With the substitution \(y=\lambda x\),
\(\seteqnumber{0}{2.}{13}\)\begin{align*} \E [X^2] &= \int _{0}^{\infty }\lambda x^2 \exp (-\lambda x)\, dx \\ &= \frac {1}{\lambda ^2} \bigg [-2e^{-y}-2ye^{-y}-y^2e^{-y} \bigg ]_{0}^{\infty } = \frac {2}{\lambda ^2} \end{align*}
Mode: \(x = 0\)
Median: \(x = \dfrac {\ln (2)}{\lambda }\)
-
Example 2.3: A model for the occurrence of failures in a certain system is described by the distribution \(X\sim \mathrm {Exp}(\lambda )\), where \(\lambda =\frac {1}{1000} \left [\frac {1}{\text {hour}}\right ]\) (note that \(\lambda \) carries physical units). Questions:
-
(a) What is the mean time to failure?
-
(b) What is the probability of a failure in the time interval \([1000,\infty ]\)?
-
(c) What is the probability of a failure in the first 100 hours of operation?
-
(d) What is the probability of a failure after 100 hours and before 1000 hours (the time interval \([100,1000]\))?
-
(e) What is the time span for which the probability of a failure is \(\frac {1}{2}\)?
-
Solution: The random variable \(X\) describes the time to failure.
\(\seteqnumber{0}{2.}{14}\)\begin{align*} \E [X] &= \frac {1}{\lambda } = 1000\ [\text {hours}]\\ \Pr (X>1000) &= 1- F_X(1000) = 1- \left [1 - \exp \left (-\tfrac {1000}{1000}\right )\right ] \approx 0.36\\ \Pr (X\le 100) &=F_X(100) \approx 0.0953\\ \Pr (100\le X \le 1000) & = F_X(1000) - F_X(100)\\ &= 0.6321 - 0.0953 \approx 0.537\\ \Pr (X \le x) &= \Pr (X > x) = \frac {1}{2} \\ \Rightarrow F_X(x) &= 1 - F_X(x) = \frac {1}{2}\\ \Rightarrow x &= \frac {\ln \left (\frac {1}{2}\right )}{-\frac {1}{1000}} =\frac {\ln (2)}{\lambda }\approx 693\ [\text {hours}] \end{align*}
-
Relation to the geometric distribution: If \(X\sim \mathrm {Exp}(\lambda )\) and we set
\[Y = \floor {X},\]
then \(Y\sim \mathrm {Geo}(1-\exp (-\lambda ))\), where \(\floor {\cdot }\) denotes rounding down to the nearest integer.
2.2.2 Uniform Distribution
PDF: For \(X\sim \mathrm {U}[a,b]\) we have (Fig. 2.1a)
\(\seteqnumber{0}{2.}{14}\)\begin{equation} f_X(x) = \begin{cases} \dfrac {1}{b-a} & a\le x\le b\\ 0 &\text {otherwise} \end {cases} \end{equation}
For \(X\sim \mathrm {U}(a,b)\), replace \(\le \) with \(<\).
CDF: For \(X\sim \mathrm {U}[a,b]\) we have (Fig. 2.1b)
\(\seteqnumber{0}{2.}{15}\)\begin{equation} F_X(x) = \int _{-\infty }^{x}f_X(x)\,dx=\begin{cases} 0 & x<a\\ \int _{a}^{x}\frac {1}{b-a}\,dx = \dfrac {x-a}{b-a} & a \le x < b \\ 1 & b\le x \end {cases} \end{equation}
A useful shortcut in many exercises, following from the definition of the variance,
\(\seteqnumber{0}{2.}{19}\)\begin{equation*} \E [X^2] = \Var [X] + \bigl (\E [X]\bigr )^2 \end{equation*}
The examples above summarize the properties of the uniform distribution.
-
\(\seteqnumber{0}{2.}{20}\)
\begin{align*} X &\sim \mathrm {U}[0,2]\\ Y &= X^2 \end{align*} compute \(\E [Y],\Var [Y]\).
-
\(\seteqnumber{0}{2.}{20}\)
\begin{align*} \E [Y] &= \E [X^2] = \Var [X] + \bigl (\E [X]\bigr )^2\\ &=\int _{0}^{2}x^2\frac {1}{2-0}\,dx = \frac {4}{3}\\ \E [Y^2] &= \E [X^4] = \int _{0}^{2}x^4\frac {1}{2-0}\,dx = \frac {1}{2}\frac {x^5}{5}\Big |_0^2 = \frac {16}{5}\\ \Var [Y]&= \E [Y^2] - \bigl (\E [Y]\bigr )^2 = \frac {16}{5}-\left (\frac {4}{3}\right )^2 = \frac {64}{45} \end{align*}
-
\(\seteqnumber{0}{2.}{20}\)
The uniform distribution also has a discrete version (such as the description of the outcome of a die roll).
2.2.3 Gaussian Distribution
PDF: For a Gaussian random variable \(Y\sim \mathcal {N}(\mu ,\sigma ^2)\), with expectation \(\mu \) and variance \(\sigma ^2\), we have (Fig. 2.2a, for \(\mu =0,\sigma =1\)).
\(\seteqnumber{0}{2.}{20}\)\begin{equation} f_X(x) = \dfrac {1}{\sigma \sqrt {2\pi }}\exp \left [{-\dfrac {\left (x-\mu \right )^2}{2\sigma ^2}}\right ] \end{equation}
CDF:
-
• It can only be computed numerically.
-
• For the special case of \(\mathcal {N}(0,1)\), it is denoted by \(\Phi (x)\) (Fig. 2.2b).
Adding a constant and multiplying by a constant:
\(\seteqnumber{1}{2.22}{0}\)\begin{align} X\sim \mathcal {N}(0,b) &\Rightarrow \sigma X \sim \mathcal {N}(0,b\sigma ^2)\\ X\sim \mathcal {N}(a,1) &\Rightarrow \mu + X \sim \mathcal {N}(a+\mu ,1)\\ X\sim \mathcal {N}(0,1) &\Rightarrow \mu + \sigma X \sim \mathcal {N}(\mu ,\sigma ^2) \end{align}
Transformation to \(\mathcal {N}(0,1)\):
\(\seteqnumber{0}{2.}{22}\)\begin{align} Y \sim \mathcal {N}(\mu ,\sigma ^2) \Rightarrow \frac {Y-\mu }{\sigma } \sim \mathcal {N}(0,1),\quad \Pr (Y < a) = \Phi \left (\frac {a-\mu }{\sigma }\right ) \end{align}
-
Proof.
\(\seteqnumber{0}{2.}{23}\)\begin{align} Y-\mu &\sim \mathcal {N}(0,\sigma ^2)\\ \frac {1}{\sigma }(Y-\mu ) &\sim \mathcal {N}(0,1) \end{align}
Q-function: For \(X\sim \mathcal {N}(0,1)\), we can define (a very useful alternative to the CDF)
\(\seteqnumber{0}{2.}{25}\)\begin{equation} Q(x) = \Pr (x < X) = \frac {1}{\sqrt {2\pi }}\int _{x}^\infty \exp \left (-\frac {s^2}{2}\right )ds. \end{equation}
Properties of the Q-function:
\(\seteqnumber{1}{2.27}{0}\)\begin{align} Q(x) &= 1-\Phi (x) = \Phi (-x)\\ Q(-x) &= 1- Q(x)\\ Q\left (0\right ) &= \frac {1}{2} \end{align}
For arbitrary expectation and variance: Given a Gaussian random variable \(Y\sim \mathcal {N}(\mu ,\sigma ^2)\),
\(\seteqnumber{0}{2.}{27}\)\begin{equation} \Pr (Y>y) = Q\left (\dfrac {y-\mu }{\sigma }\right ) \end{equation}
-
Example 2.9: Given the relation
\(\seteqnumber{0}{2.}{28}\)\begin{equation*} Y = X + a, \end{equation*}
where \(a\) is a constant and \(X\sim \mathcal {N}(0,\sigma ^2)\), what is the probability \(\Pr (Y \le 0)\)?
-
Solution: Following Fig. 2.3,
\(\seteqnumber{0}{2.}{28}\)\begin{equation*} \begin{aligned} Y &\sim \mathcal {N}(a,\sigma ^2)\\ \Pr (Y < 0) &=F_Y(0) = \Phi \left (\frac {0 - a}{\sigma }\right ) = \Phi \left (-\frac {a}{\sigma }\right )\\ &= Q\left (\frac {a}{\sigma }\right ) \end {aligned} \end{equation*}
An alternative numerical solution is obtained by performing a sufficiently large number of random experiments. Steps:
-
• draw \(N\) values of the random variable \(X\).
-
• compute \(Y\).
-
• count the number of experiment outcomes that satisfy the criterion \(Y<0\). Denote the result by \(n\).
-
• \(\Pr (Y < 0) \approx \dfrac {n}{N}\).
-
-
Central limit theorem: Given \(n\) independent, identically distributed random variables of any distribution \(X_1,X_2,\ldots ,X_n\), with expectation \(\mu \) and variance \(\sigma ^2\). According to the theorem, the mean is Gaussian distributed
\(\seteqnumber{0}{2.}{28}\)\begin{equation} \lim \limits _{n\rightarrow \infty } \frac {1}{n}\sum _{k=1}^{n}X_k \sim \mathcal {N}\left (\mu ,\sigma ^2\right ) \end{equation}
In practice, for \(n\gtrsim 30\), the theorem holds to a reasonable approximation for any type of distribution, including discrete distributions.
2.2.4 Chi-square Distribution
Chi-square variable: Let \(Z_1, Z_2, \ldots , Z_k\) be independent standard normal variables, \(Z_i\sim \mathcal {N}(0,1)\). Their sum of squares
\(\seteqnumber{0}{2.}{29}\)\begin{equation} \label {eq-chi2-def} Q = \sum _{i=1}^{k} Z_i^2 \end{equation}
follows the chi-square distribution with \(k\) degrees of freedom, written \(Q\sim \chi ^2_k\).
Range and degrees of freedom: Two properties follow directly from (2.30): \(Q\ge 0\), because it is a sum of squares, and the parameter \(k\) is nothing more than the number of independent squared terms being added.
PDF: The probability density function is (Fig. 2.4b)
\(\seteqnumber{0}{2.}{30}\)\begin{equation} f(q;k) = \frac {1}{2^{k/2}\,\Gamma (k/2)}\,q^{k/2-1}e^{-q/2},\qquad q>0, \end{equation}
where \(\Gamma (\cdot )\) is the gamma function. The density is right-skewed, and the skew decreases as \(k\) grows.
Moments:
\(\seteqnumber{0}{2.}{31}\)\begin{align} \E [Q] &= k,\\ \Var [Q] &= 2k. \end{align}
Additivity: For independent \(Q_1\sim \chi ^2_{k_1}\) and \(Q_2\sim \chi ^2_{k_2}\),
\(\seteqnumber{0}{2.}{33}\)\begin{equation} \label {eq:rand1:chi2-additivity} Q_1 + Q_2 \sim \chi ^2_{k_1+k_2}. \end{equation}
The degrees of freedom add exactly as the squared terms of (2.30) do.
Combined with the central limit theorem (Feature 2.2.3), additivity also explains the large-\(k\) behavior: a \(\chi ^2_k\) variable is a sum of \(k\) independent contributions, so
\(\seteqnumber{0}{2.}{34}\)\begin{equation} Q \;\approx \; \mathcal {N}(k,\,2k), \qquad k \gg 1. \end{equation}
Upper-tail quantile: The upper-tail quantile \(\chi ^2_{k,\,1-\alpha }\) is the value that a \(\chi ^2_k\) variable exceeds with probability \(\alpha \) (Fig. 2.4a),
\(\seteqnumber{0}{2.}{35}\)\begin{equation} \label {eq-chi2-quantile} \Pr \left (Q > \chi ^2_{k,\,1-\alpha }\right ) = \alpha , \qquad Q\sim \chi ^2_k , \end{equation}
the subscript naming the cumulative level \(1-\alpha \) rather than the tail left above it. It is computed numerically in general; two cases are closed form.
Closed form for \(k=1\): By (2.30) a \(\chi ^2_1\) variable is \(Z^2\) with \(Z\sim \mathcal {N}(0,1)\), so its quantile is a two-sided normal threshold squared,
\(\seteqnumber{0}{2.}{36}\)\begin{equation} \label {eq-chi2-quantile-1} \chi ^2_{1,\,1-\alpha } = \left [\Phi ^{-1}\left (1-\alpha /2\right )\right ]^2 , \end{equation}
which returns \(3^2 = 9\) at \(\alpha = 0.0027\).
Closed form for \(k=2\): For \(k=2\) the density reduces to \(f(q;2) = \tfrac {1}{2}e^{-q/2}\), so the tail and its inverse are elementary,
\(\seteqnumber{0}{2.}{37}\)\begin{equation} \label {eq-chi2-tail-2} \Pr \left (\chi ^2_2 > \tau \right ) = e^{-\tau /2} , \qquad \chi ^2_{2,\,1-\alpha } = -2\ln \alpha . \end{equation}
Sum of squared deviations: For an i.i.d. Gaussian sample \(X_1,\ldots ,X_n\sim \mathcal {N}(\mu ,\sigma ^2)\), the sum of squared deviations around the sample mean \(\bar X\) satisfies
\(\seteqnumber{0}{2.}{38}\)\begin{equation} \label {eq-chi2-ssd} \frac {1}{\sigma ^2}\sum _{i=1}^{n}\left (X_i-\bar X\right )^2 \sim \chi ^2_{n-1}. \end{equation}
-
Proof. Add and subtract \(\bar X\) inside each square of \(\sum _i (X_i-\mu )^2\). The cross term vanishes, because the deviations around the sample mean satisfy \(\sum _i (X_i-\bar X)=0\), which leaves the identity
\(\seteqnumber{0}{2.}{39}\)\begin{equation*} \sum _{i=1}^{n}\left (X_i-\mu \right )^2 = \sum _{i=1}^{n}\left (X_i-\bar X\right )^2 + n\left (\bar X-\mu \right )^2 . \end{equation*}
Dividing throughout by \(\sigma ^2\) puts all three terms in standardized form,
\(\seteqnumber{0}{2.}{39}\)\begin{equation*} \sum _{i=1}^{n}\left (\frac {X_i-\mu }{\sigma }\right )^2 = \frac {1}{\sigma ^2}\sum _{i=1}^{n}\left (X_i-\bar X\right )^2 + \left (\frac {\bar X-\mu }{\sigma /\sqrt {n}}\right )^2 . \end{equation*}
Two of the three are already known. The left-hand side is a sum of \(n\) squared standard normals, so by (2.30) it is \(\chi ^2_n\). The right-most term is the square of a single standard normal, because \(\bar X\sim \mathcal {N}(\mu ,\sigma ^2/n)\), so it is \(\chi ^2_1\). The two terms on the right are independent, since for a Gaussian sample \(\bar X\) and the deviations \(X_i-\bar X\) are.
Now read Eq (2.34) backwards. A \(\chi ^2_n\) variable has been written as a sum of two independent chi-square terms, one of which is \(\chi ^2_1\). Degrees of freedom add, so the remaining term must carry \(n-1\) of them, which is (2.39).
2.2.5 Student t-Distribution
Student t-distribution: Let \(Z\sim \mathcal {N}(0,1)\) and \(Q\sim \chi ^2_k\) be independent. The ratio
\(\seteqnumber{0}{2.}{39}\)\begin{equation} \label {eq-t-def} \frac {Z}{\sqrt {Q/k}} \end{equation}
follows the Student t-distribution with \(k\) degrees of freedom, written \(t_k\).
Properties: The density is symmetric about zero, as \(\mathcal {N}(0,1)\) is, but with heavier tails (Fig. 2.5). The extra weight comes from the denominator of (2.40), which is itself random: an unusually small \(Q\) inflates the ratio even when \(Z\) is ordinary. As \(k\) grows the denominator concentrates around \(1\), and
\(\seteqnumber{0}{2.}{40}\)\begin{equation} t_k \longrightarrow \mathcal {N}(0,1), \qquad k\rightarrow \infty . \end{equation}
Upper-tail quantile: As for the chi-square, \(t_{k,\,1-\alpha /2}\) is the value that a \(t_k\) variable exceeds with probability \(\alpha /2\),
\(\seteqnumber{0}{2.}{41}\)\begin{equation} \label {eq-t-quantile} \Pr \left (t_k > t_{k,\,1-\alpha /2}\right ) = \frac {\alpha }{2} . \end{equation}
Because the density is symmetric, the same value serves both tails: an interval of total probability \(1-\alpha \) runs from \(-t_{k,\,1-\alpha /2}\) to \(+t_{k,\,1-\alpha /2}\).
Standardized sample mean: Let \(x_1,\ldots ,x_n\) be i.i.d. \(\mathcal {N}(\mu ,\sigma ^2)\), with sample mean \(\bar x\) and unbiased sample std \(s_x\). Then
\(\seteqnumber{0}{2.}{42}\)\begin{equation} \label {eq-t-sample-mean} \frac {\bar x - \mu }{s_x/\sqrt {n}} \sim t_{n-1} . \end{equation}
-
Proof. Both ingredients of (2.40) are already available.
The numerator. The sample mean of \(n\) Gaussian observations is itself Gaussian, \(\bar x\sim \mathcal {N}(\mu ,\sigma ^2/n)\), so standardizing it by its own std gives
\(\seteqnumber{0}{2.}{43}\)\begin{equation*} Z = \frac {\bar x - \mu }{\sigma /\sqrt {n}} \sim \mathcal {N}(0,1) . \end{equation*}
The denominator. By (2.39),
\(\seteqnumber{0}{2.}{43}\)\begin{equation*} Q = \frac {1}{\sigma ^2}\sum _{i=1}^{n}\left (x_i-\bar x\right )^2 \sim \chi ^2_{n-1} . \end{equation*}
The unbiased variance is \(s_x^2 = \frac {1}{n-1}\sum _{i=1}^{n}(x_i-\bar x)^2\), so this same \(Q\) equals \((n-1)s_x^2/\sigma ^2\), that is
\(\seteqnumber{0}{2.}{43}\)\begin{equation*} s_x = \sigma \sqrt {\frac {Q}{n-1}} . \end{equation*}
Independence. For a Gaussian sample the sample mean \(\bar x\) and the deviations \(x_i-\bar x\) are independent, hence so are \(Z\) and \(Q\). This is the condition (2.40) requires of its two ingredients.
Combining. Divide the numerator and the denominator of the left-hand side of (2.43) by \(\sigma \) and substitute both results,
\(\seteqnumber{0}{2.}{43}\)\begin{equation*} \frac {\bar x - \mu }{s_x/\sqrt {n}} = \frac {\left (\bar x - \mu \right )/\left (\sigma /\sqrt {n}\right )}{s_x/\sigma } = \frac {Z}{\sqrt {Q/(n-1)}} , \end{equation*}
which is (2.40) with \(k=n-1\). The unknown \(\sigma \) has cancelled, which is the whole point: the left-hand side is computable from the sample alone.
The degree of freedom lost here is the same one that makes \(s_x\) unbiased.
Estimating \(\sigma \) instead of knowing it therefore costs accuracy, and the cost is visible in the quantile. At \(\alpha =0.05\) the value of \(t_{k,\,0.975}\) is \(2.571\) for \(k=5\), \(2.228\) for \(k=10\), \(2.042\) for \(k=30\) and \(1.984\) for \(k=100\), against \(\Phi ^{-1}(0.975)=1.960\) for a known \(\sigma \). Past roughly \(k=30\) the gap is under \(5\%\), which is where the usual “\(n\gtrsim 30\)” rule of thumb comes from.
2.3 Exercises
2.1 Graphs of the CDF of Gaussian random variables are given.
\(\seteqnumber{0}{2.}{43}\)\begin{align*} X &\sim \mathcal {N}(\mu _X,\sigma _X^2) \\ Y &\sim \mathcal {N}(\mu _Y,\sigma _Y^2) \\ Z &\sim \mathcal {N}(\mu _Z,\sigma _Z^2) \end{align*}
-
(a) State the expectation of each of the variables, \(\mu _X,\mu _Y,\mu _Z\). Justify your answer!
-
(b) Order the graphs by the value of their standard deviation \(\sigma _i>\sigma _j>\sigma _k,\;\; i,j,k\in \left \lbrace X,Y,Z \right \rbrace \), in decreasing order. Justify your answer!
2.2 (Sum of many variables) Consider a sequence of \(200\) independent experiments \(X_1,X_2,\ldots ,X_{200}\), each distributed as
\[ X_i\sim \mathcal {N}\bigl (\tfrac 12,\,(\tfrac 12)^2\bigr ). \]
Compute the probability \(\Pr (90 \le Y \le 110)\) for the sum
\[ Y = X_1 + X_2 + \cdots + X_{200}. \]
2.3 (includes simulation) Given the random variables,
\(\seteqnumber{0}{2.}{43}\)\begin{align} X& \sim \mathrm {Exp}(\lambda ) \\ Y&=X^2+1 \end{align}
-
(a) Compute \(\E [Y],\Var [Y]\).
-
(b) For \(\lambda = 1/2\), plot \(\E [Y],\Var [Y]\) computed by numerical simulation, as a function of the number of random experiments \(N\le 10^6\). Use a logarithmic axis for \(N\). Compare the values obtained with the analytic results from the previous part.
-
(c) Plot the resulting PDF of \(Y\) using a histogram.
2.4 (simulation) Draw a sequence of numbers with a Bernoulli distribution, \(X\sim \mathrm {Ber}(p)\). Count the number of consecutive draws until the first “1” appears. Plot the distribution (PDF) of the resulting number of draws. Compare the result to the PDF of a geometric distribution.
2.5 (simulation) Given \(X\sim \mathrm {U}[0,1]\). Show by simulation the relation
\(\seteqnumber{0}{2.}{45}\)\begin{equation*} \Pr \left (X^{\frac {1}{4}}<\frac {1}{2}\right ) = \frac {1}{16} = 0.0625 \end{equation*}