Perché il test delle ipotesi di base si concentra sulla media e non sulla mediana?

Nei corsi di statistica di base under-grad, agli studenti viene (di solito?) Insegnato test di ipotesi per la media di una popolazione.
Perché l'attenzione è rivolta alla media e non alla mediana? La mia ipotesi è che sia più facile testare la media a causa del teorema del limite centrale, ma mi piacerebbe leggere alcune spiegazioni colte.

— nafrtiti
fonte

The mean has useful properties for uniqueness, calculation, and calculus. It is often related to the sufficient statistics.

— Henry

Risposte:

Because Alan Turing was born after Ronald Fisher.

In the old days, before computers, all this stuff had to be done by hand or, at best, with what we would now call calculators. Tests for comparing means can be done this way - it's laborious, but possible. Tests for quantiles (such as the median) would be pretty much impossible to do this way.

For example, quantile regression relies on minimizing a relatively complicated function.This would not be possible by hand. It is possible with programming. See e.g. Koenker or Wikipedia.

Quantile regression has fewer assumptions than OLS regression and provides more information.

— Peter Flom - Reinstate Monica
fonte

At that time computers did exist but meant something very different from what we mean with it now.

— Maarten Buis

Indeed! Computers were people who did the calculations.

— Peter Flom - Reinstate Monica

@nafrtiti The syllabus is changing, but slowly. There is a lot of momentum to overcome and people outside statistics are not used to the new ideas so may reject them.

— Peter Flom - Reinstate Monica

@SunQingyao Sorting is much more expensive than adding. Adding is O(n) and it's one of the most basic operations of hardware and requires only one register. In addition to that, all I need to know is the total and the number of items to more data and calculate the new mean. In order to calculate the median, I need the entire set

— JimmyJames

With Quick select (and using median-of-5 to select pivot if bad pivots are random chosen) you can find a quantile in O(N), making the gap between median and average smaller. Of course you need to know that such methods exists (which was unknown even at Turings time).

— Surt

I would like to add a third reason to the correct reasons given by Harrell and Flom. The reason is that we use Euclidean distance (or L2) and not Manhattan distance (or L1) as our standard measure of closeness or error. If one has a number of data points $x_1, \ldots x_n$ and one wants a single number $\theta$ to estimate it, an obvious notion is to find the number that minimizes the 'error' that number creates the smallest difference between the chosen number and the numbers that constitute the data. In mathematical notation, for a given error function E, one wants to find $min_{\theta \in \Bbb{R}} (E(\theta,x_1, \ldots x_n) = min_{\theta \in \Bbb{R}}(\sum_{i=1}^{i=n} E(\theta,x_i))$ . If one takes for E(x,y) the L2 norm or distance, that is $E(x,y) = (x-y)^2$ then the minimizer over all $\theta \in \Bbb{R}$ is the mean. If one takes the L1 or Manhattan distance, the minimizer over all $\theta \in \Bbb{R}$ is the median. Thus the mean is the natural mathematical choice - if one is using L2 distance !

— aginensky
fonte

Since

E

$E$ is broadly used to denote expectation, I suggest replacing

E

$E$ with, say,

Err

$\text{Err}$ .

— Richard Hardy

Perhaps it is worth noting that

x^{2}

$x^2$ is differentiable at

x = 0

$x=0$ while

| x |

$|x|$ is not. In my opinion, this is a subtle but key underlying reason why MSE is more prevalent in the mathematical statistics arena than MAE.

— Just_to_Answer

@Just_to_Answer - I think that is yet another reason-sort of. I've thought about this a lot over the years. For me, I've concluded that the what you say is tied up with why we generally use Euclidean and not Manhattan distance :)

— aginensky

Often the mean is chosen over the median not because it's more representative, robust, or meaningful but because people confuse estimator with estimand. Put another way, some choose the population mean as the quantity of interest because with a normal distribution the sample mean is more precise than the sample median. Instead they should think more, as you have done, about the true quantity of interest.

One sidebar: we have a nonparametric confidence interval for the population median but there is no nonparametric method (other than perhaps the numerically intensive empirical likelihood method) to get a confidence interval for the population mean. If you want to stay distribution-free you might concentrate on the median.

Note that the central limit theorem is far less useful than it seems, as been discussed elsewhere on this site. It effectively assumes that the variance is known or that the distribution is symmetric and has a shape such that the sample variance is a competitive estimator of dispersion.

— Frank Harrell
fonte

I believe it's possible to construct a nonparametric confidence interval for the mean - say via a permutation test (this can be done under an assumption of symmetry without assuming any specific functional form, for example). That's a somewhat restricted situation, though it's also possible under some other assumptions than symmetry. If you're prepared to deal with the approximate coverage that comes with bootstrapping one can get nonparametric intervals without assumptions like symmetry.

— Glen_b -Reinstate Monica

If it assumes symmetry it is parametric. Haven't seen this extended to non-symmetric cases. The bootstrap (all variants except perhaps the studentized t method) is extremely inaccurate under severe asymmetry. See stats.stackexchange.com/questions/186957

— Frank Harrell

Symmetry is not finite-parametric. A Wilcoxon signed rank test assumes symmetry (in order to have exhangeability of signs) under the null. You'd call that parametric?

— Glen_b -Reinstate Monica

stats.stackexchange.com/questions/9573 stats.stackexchange.com/questions/186957

— Frank Harrell

On @Glen_b question about symmetry - that's an excellent question. The Wilcoxon signed-rank test is an interesting case because, unlike the WIlcoxon 2-sample test, makes a heavy symmetry assumption. I guess you could say that you can be non-parametric while still requiring some kind of general assumption such as symmetry. Maybe the terminology should be "nonparametric with restrictions"? On the other hand the nonparametric 2-sample test has restrictions with regard to what optimizes type II error (but not type I error).

— Frank Harrell