Stimatori della massima verosimiglianza per una distribuzione troncata

Considerare campioni indipendenti ottenuta da una variabile casuale che si presume seguire una distribuzione troncata (ad esempio un tronco distribuzione normale ) di nota (finito) valori minimo e massimo e ma di parametri ignoti e . Se seguisse una distribuzione non troncata, gli stimatori della massima verosimiglianza e per e da sarebbero la media del campione $N$ $S$ $X$ $a$ $b$ $\mu$ $\sigma^2$ $X$ $\widehat\mu$ $\widehat\sigma^2$ $\mu$ $\sigma^2$ $S$ $\widehat\mu = \frac{1}{N} \sum_i S_i$ e la varianza di esempio . Tuttavia, per una distribuzione troncata, la varianza del campione definita in questo modo è limitata da quindi non è sempre uno stimatore coerente: per , non può convergere in probabilità in come va all'infinito. Quindi sembra che e non siano gli stimatori della massima verosimiglianza di e per una distribuzione troncata. Naturalmente, questo è prevedibile dal momento che e $\widehat\sigma^2 = \frac{1}{N} \sum_i (S_i - \widehat\mu)^2$ $(b-a)^2$ $\sigma^2 > (b-a)^2$ $\sigma^2$ $N$ $\widehat\mu$ $\widehat\sigma^2$ $\mu$ $\sigma^2$ $\mu$ $\sigma^2$ i parametri di una distribuzione normale troncata non sono la media e la varianza.

Quindi, quali sono gli stimatori della massima verosimiglianza di e $\mu$ $\sigma$ parameters of a truncated distribution of known minimum and maximum values?

— a3nm
fonte

Are you sure about your analysis? I think you are making an invalid assumption: for the truncated situation, the MLE of

σ^{2}

$\sigma^2$ is no longer the sample variance (and, in general, the MLE of

μ

$\mu$ is no longer the sample mean)!

— whuber

whuber: I know, this is precisely my question: what are the MLEs of

σ^{2}

$\sigma^2$ and

μ

$\mu$ in the truncated case? Adding a sentence to insist on this.

— a3nm

There isn't a closed form solution. All you can do is numerically minimize the log likelihood. But this is qualitatively no different than many other models, such as logistic regression, which also have no closed form solution.

— whuber

whuber: If this is true, this is pretty disappointing. Do you have references about the lack of closed form solutions? Are there closed-form estimators that are not maximum likelihood but are at least consistent (and optionally unbiased?).

— a3nm

@whuber: Can you at least simplify your samples into sufficient statistics so that the minimization is fast?

— Neil G

Consider any location-scale family determined by a "standard" distribution $F$ ,

Ω_{F} = {F_{(μ, σ)} : x \to F (\frac{x - μ}{σ}) ∣ σ > 0} .

$\Omega_F = \left\{F_{(\mu, \sigma)}: x \to F\left(\frac{x-\mu}{\sigma}\right) \mid \sigma \gt 0\right\}.$

Assuming $F$ differentiable we readily find that the PDFs are $\frac{1}{\sigma}f\left((x-\mu)/\sigma\right)dx$ .

Truncating these distributions to restrict their support between $a$ and $b$ , $a \lt b$ , means that the PDFs are replaced by

f_{(μ, σ; a, b)} (x) = \frac{f (\frac{x - μ}{σ}) d x}{σ C (μ, σ, a, b)}, a \leq x \leq b

$f_{(\mu, \sigma; a,b)}(x) = \frac{f\left(\frac{x-\mu}{\sigma}\right)dx}{\sigma C(\mu, \sigma, a, b)}, a \le x \le b$

(and are zero for all other values of $x$ ) where $C(\mu, \sigma, a, b) = F_{(\mu,\sigma)}(b) - F_{(\mu,\sigma)}(a)$ is the normalizing factor needed to ensure that $f_{(\mu, \sigma; a, b)}$ integrates to unity. (Note that $C$ is identically $1$ in the absence of truncation.) The log likelihood for iid data $x_i$ therefore is

Λ (μ, σ) = \sum_{i} [\log f (\frac{x_{i} - μ}{σ}) - \log σ - \log C (μ, σ, a, b)] .

$\Lambda(\mu, \sigma) = \sum_i \left[\log{f\left(\frac{x_i-\mu}{\sigma}\right)} - \log{\sigma}-\log{C(\mu, \sigma, a, b)}\right].$

Critical points (including any global minima) are found where either $\sigma=0$ (a special case I will ignore here) or the gradient vanishes. Using subscripts to denote derivatives, we may formally compute the gradient and write the likelihood equations as

\begin{aligned} 0 & = \frac{\partial Λ}{\partial μ} & = \sum_{i} [\frac{- f_{μ} (\frac{x_{i} - μ}{σ})}{f (\frac{x_{i} - μ}{σ})} - \frac{C_{μ} (μ, σ, a, b)}{C (μ, σ, a, b)}] \\ 0 & = \frac{\partial Λ}{\partial σ} & = \sum_{i} [\frac{- f_{σ} (\frac{x_{i} - μ}{σ})}{σ^{2} f (\frac{x_{i} - μ}{σ})} - \frac{1}{σ} - \frac{C_{σ} (μ, σ, a, b)}{C (μ, σ, a, b)}] \end{aligned}

$\eqalign{ 0 &= \frac{\partial\Lambda}{\partial\mu} &= \sum_i \left[\frac{-f_\mu\left(\frac{x_i-\mu}{\sigma}\right)}{f\left(\frac{x_i-\mu}{\sigma}\right)} -\frac{C_\mu(\mu,\sigma,a,b)}{C(\mu,\sigma,a,b)}\right] \\ 0 &= \frac{\partial\Lambda}{\partial\sigma} &= \sum_i \left[\frac{-f_\sigma\left(\frac{x_i-\mu}{\sigma}\right)}{\sigma^2f\left(\frac{x_i-\mu}{\sigma}\right)} -\frac{1}{\sigma}-\frac{C_\sigma(\mu,\sigma,a,b)}{C(\mu,\sigma,a,b)}\right] }$

Because $a$ and $b$ are fixed, drop them from the notation and write $nC_\mu(\mu, \sigma, a, b)/C(\mu, \sigma,a,b)$ as $A(\mu,\sigma)$ and $nC_\sigma(\mu, \sigma, a, b)/C(\mu, \sigma,a,b)$ as $B(\mu, \sigma)$ . (With no truncation, both functions would be identically zero.) Separating the terms involving the data from the rest gives

\begin{aligned} - A (μ, σ) & = \sum_{i} \frac{f_{μ} (\frac{x_{i} - μ}{σ})}{f (\frac{x_{i} - μ}{σ})} \\ - σ^{2} B (μ, σ) - n σ & = \sum_{i} \frac{f_{σ} (\frac{x_{i} - μ}{σ})}{f (\frac{x_{i} - μ}{σ})} \end{aligned}

$\eqalign{ -A(\mu,\sigma) &= \sum_i \frac{f_\mu\left(\frac{x_i-\mu}{\sigma}\right)}{f\left(\frac{x_i-\mu}{\sigma}\right)} \\ -\sigma^2 B(\mu,\sigma) - n\sigma &= \sum_i \frac{f_\sigma\left(\frac{x_i-\mu}{\sigma}\right)}{f\left(\frac{x_i-\mu}{\sigma}\right)} }$

By comparing these to the no-truncation situation it is evident that

Any sufficient statistics for the original problem are sufficient for the truncated problem (because the right hand sides have not changed).
Our ability to find closed-form solutions relies on the tractability of $A$ and $B$ . If these do not involve $\mu$ and $\sigma$ in simple ways, we cannot hope to obtain closed-form solutions in general.

For the case of a normal family, $C(\mu,\sigma,a,b)$ of course is given by the cumulative normal PDF, which is a difference of error functions: there is no chance that a closed-form solution can be obtained in general. However, there are only two sufficient statistics (the sample mean and variance will do) and the CDF is as smooth as can be, so numerical solutions will be relatively easy to obtain.

— whuber
fonte

Thanks a lot for this very detailed answer! I'm not sure I get what

f_{μ}

$f_\mu$ ,

f_{σ}

$f_\sigma$ ,

C_{μ}

$C_\mu$ , and

C_{σ}

$C_\sigma$ are, could you define them? Also, it's obvious but to be precise maybe you could say that your expression for the pdf is for

x \in [a, b]

$x \in [a, b]$ (and the pdf is zero outside of that). Thanks again!

— a3nm

The usual longer notation is

C_{μ} = \frac{\partial}{\partial μ} C (μ, σ, a, b)

$C_\mu = \frac{\partial}{\partial\mu}C(\mu,\sigma,a,b)$ , etc: as announced, it is a derivative. I will make the second change you suggest because it's an important clarification, thanks.

— whuber

Also, since your answer is more general than the one I expected, I edited my question to insist less on the case of normal distributions. Thanks again for your effort.

— a3nm

It was easier to explain at this level of generality compared to focusing on the Normal distributions! Computing the derivatives and showing the precise form of the CDF are unnecessary distractions (although useful when you start actually coding the numerical solution).

— whuber

Thanks for fixing! You missed one of them; could you review my edit?

— a3nm