[barmar]

helene_t, on Oct 1 2008, 01:16 PM, said:

Modes, on the other hand, I never found a use for, but that is no doubt just related to the limited scope of my work.

But it comes up in duplicate bridge. The "field result" is probably the mode.

[barmar]

We have software at my company that makes use of median, and this issue has recently come up. We do network server load balancing and failover, and we have monitoring systems that test the servers to see if they're alive and responding in a reasonable time. 6-8 systems perform a test, and we use the median response time, arbitrarily assigning a very large number (75.0) if the test fails. It's not uncommon for the data set to look something like:

0.01 0.01 0.02 0.03 0.5 0.6 75.0 75.0

If the mean is below 4, the server is alive, otherwise it's considered down (it's actually more complicated than that, but the details aren't important).

We're phasing in a new version of the software. The old version always used the lower of the two middle values when there were an even number, the new one uses the mean of them. It's not too unusual for exactly half the values to be the 75.0 penalty value, while the others are tiny (because customers sometimes fail to update their firewalls when we announce new testing IPs). When this happens, the old version would declare it up, but the new one will calculate median = 37.x, so it's considered down.

We've decided not to fix this incompatibility. If 4 test machines are failing, it could just as easily have been 5, so it's an arbitrary distinction to make. The primary purpose of using the median is to throw out real outliers, like in Gnome's example. If the data set is split half and half between two types of values, neither one can really be considered outlying.

[hrothgar]

The set of assumptions required to provide that the mean is the Best Linear Unbiased Estimator (BLUE) is fairly restrictive. You often need to break out medians and IQR's to avoid using a (potentially) biased estimator.

I used to run into this issue all the time studying packet delays on TCP/IP networks...

[kenberg]

Elianna, on Oct 1 2008, 09:30 AM, said:

kenberg, on Oct 1 2008, 05:41 AM, said:

For example, suppose that we have an odd number of positive data points x_1,x_2, ... with  a median value m. Suppose now we square those numbers, y=x^2, getting data points y_1,y_2 ... . The median of this new data set will be m^2. TIf we have an even number of data points x_i with median interval [a,b] and do the same maneuver, the median of the y_i would be the interval [a^2,b^2].

Isn't that only true if the x_i's are positive?

yes. I said so I think.

[kenberg]

Roupoil, on Oct 1 2008, 11:37 AM, said:

Well, it depends on your definition of the median.

The "most mathematical" définition is to say that the median is a number which minimizes the sum of the (absolute) distances to all the numbers of your list (and the arithmetic mean to be the one which minimizes the sum of the squares of these distances), in which case all the interval between the two middle numbers is a median, so why not take the middle of this interval ?

But there are indeed people who want the median to be one of the numbers of the list, and usually pick the smaller one. A convenient definition for this median is the smallest number for which the cumulate frequency reachs 0.5.

PS : Sorry for the maybe bad translation from French of some of the mathematical words...

I like this definition of the median a lot. Thanks, I had never thought of this characterization before. It displays the median as a special case of a higher dimensional problem. And it's another way of presenting the median of an even number of data points as an interval rather than a specific number (Any point in [5,7] will minimize the indicated sum if the data points are 1,5,7,12). And it nicely relates to the definition of the mean. All in all, a neat definition.

[helene_t]

barmar, on Oct 2 2008, 01:01 AM, said:

helene_t, on Oct 1 2008, 01:16 PM, said:

Modes, on the other hand, I never found a use for, but that is no doubt just related to the limited scope of my work.

But it comes up in duplicate bridge. The "field result" is probably the mode.

Sort of. At matchpoints, the single most relevant score to compare the score you might get when making a particular choice, is indeed the mode.

But ideally you should consider the whole distribution of scores at other tables Suppose the distribution is
40% : +120
30% : +90
30% : -50

Here a +120 is the mode, yet an action that changes +50 to +110 is more lucrative than one that changes +110 to +200.