[Trinidad]

helene_t, on 2013-November-07, 06:59, said:

No. Your stats were for a single session. Most of the variance is random. That 77% score less than 0.8 IMPs over a long session (Eagles' 168 boards) would only be true if the variance in your table was attributable to skill difference alone.

Even in a highly heterogenous field you wouldn't expect 23% to be able to maintain an average of 0.8 IMPs over 168 boards.

Maybe 98% of the dump idiots are dumper than Eagles

That is correct.

Probably the best way to get a feeling for your result is to convert the IMPs to victory points. This is corrected for the number of boards played.

Rik

[HighLow21]

Trinidad, on 2013-November-06, 08:32, said:

I did a quick calculation based on the Cross-Imp competition at our club.

We play 26 boards per evening. At the end of the evening, we get the results: for each pair the number of IMPs/board. I calculated the standard deviation of the IMPs per board for 7 of these evenings. It was slightly above 1.1 IMPs/board. Assuming that the skill at BBO and in our club are normally distributed and that the standard deviation of the skill in my clubs is as large as the standard deviation in skill on BBO (quite bold assumptions, but you have to assume something) this would give the following percentile table for BBO:

-2        3
-1.8      5
-1.6      7
-1.4      10
-1.2      13
-1        18
-0.8      23
-0.6      29
-0.4      35
-0.2      43
0         50
0.2       57
0.4       65
0.6       71
0.8       77
1         82
1.2       87
1.4       90
1.6       93
1.8       95
2         97

The first column shows your IMPs/board score. The second shows what percentage of the pairs are worse than you.

I wouldn't call it the exact science, but it will give you an indication.

Rik

Rik just to be clear:
- To me what this looks like is 1.1 is the standard error of the average of the IMPs, which would make sense if you were playing 20-40 boards. (As you said, 26 boards.)
- And if I multiply this by the square root of 26 I get 5.6, which would be the standard deviation of the IMPs score on any individual board. This is very consistent with my math.
- Just to be clear -- 1.1 is the standard deviation of the 7 nightly IMPs averages? Can you type in the data so I can see it?

[HighLow21]

Trinidad, on 2013-November-06, 10:52, said:

Of course, it does.



So, the table suggest that about 75% of those dumb idiots are playing worse than you and about 25% of these dumb idiots are playing better than you. So, it indicates that -for a dumb idiot- you are relatively smart.

Rik

No, the percentiles given only indicate what percent of the time you are likely to receive that score GIVEN THAT you are all equally matched. In other words, if you are evenly matched, 75% of the time you will score +0.76 IMPs average or lower. 25% of the time it would be higher than that.

It says nothing about what percent of pairs are better/worse than you.

[HighLow21]

helene_t, on 2013-November-07, 06:59, said:

No. Your stats were for a single session. Most of the variance is random. That 77% score less than 0.8 IMPs over a long session (Eagles' 168 boards) would only be true if the variance in your table was attributable to skill difference alone.

Even in a highly heterogenous field you wouldn't expect 23% to be able to maintain an average of 0.8 IMPs over 168 boards.

Maybe 98% of the dump idiots are dumper than Eagles

To go from 26 boards to 168 boards you would have to divide everything in the left-hand column by the square root of 7, which is the square root of the ratio of hands played. The variation decreases as a function of the square root of the number of observations.

Assuming Trinidad's data is correct, I've included a table here. Each row represents the number of boards, and each column represents the percent of the time you would achieve that score or lower, assuming your expected score is zero. (If your expected score is different, simply add your expected score to ALL of the entries in this table.)

As an example -- if your expect score is zero (competition is TOTALLY EVEN), you play 52 boards, and you score a +0.81 IMP average, you've done better than you would have done 85% of the time.

If your expected score is 0.5 IMPs per board, you play 8 boards, and get a score of +2.04 IMPs, you have done better than you would have done normally about 75% of the time. (1.04 IMPs plus your expected boost of 1 IMP per board.)

Alternatively -- if you have an expected score of 0.8 IMPs and play 128 boards, you will win almost 95% of the time. (This is because the 5% observation for 128 boards is -0.82, and you expect to add +0.8 on average to that total.)

I've also attached how it looks in Excel.


NUMBER OF IMPs/BOARD AVERAGE: NUMBER OF HANDS PLAYED (FIRST COLUMN) VERSUS PERCENTILE ACHIEVEMENT (TOP ROW). 50% = NORMAL/EXPECTED RESULT.

H/P 1% 2% 5% 10% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 98% 99%
1 -13.1 -11.5 -9.23 -7.19 -4.72 -3.78 -2.94 -2.16 -1.42 -0.7 0 0.7 1.42 2.16 2.94 3.78 4.72 5.81 7.19 9.23 11.5 13.1
5 -5.84 -5.15 -4.13 -3.21 -2.11 -1.69 -1.32 -0.97 -0.64 -0.32 0 0.32 0.64 0.97 1.32 1.69 2.11 2.6 3.21 4.13 5.15 5.84
8 -4.61 -4.07 -3.26 -2.54 -1.67 -1.34 -1.04 -0.76 -0.5 -0.25 0 0.25 0.5 0.76 1.04 1.34 1.67 2.06 2.54 3.26 4.07 4.61
13 -3.62 -3.19 -2.56 -1.99 -1.31 -1.05 -0.82 -0.6 -0.39 -0.2 0 0.2 0.39 0.6 0.82 1.05 1.31 1.61 1.99 2.56 3.19 3.62
16 -3.26 -2.88 -2.31 -1.8 -1.18 -0.95 -0.74 -0.54 -0.36 -0.18 0 0.18 0.36 0.54 0.74 0.95 1.18 1.45 1.8 2.31 2.88 3.26
26 -2.56 -2.26 -1.81 -1.41 -0.93 -0.74 -0.58 -0.42 -0.28 -0.14 0 0.14 0.28 0.42 0.58 0.74 0.93 1.14 1.41 1.81 2.26 2.56
32 -2.31 -2.04 -1.63 -1.27 -0.83 -0.67 -0.52 -0.38 -0.25 -0.12 0 0.12 0.25 0.38 0.52 0.67 0.83 1.03 1.27 1.63 2.04 2.31
52 -1.81 -1.6 -1.28 -1 -0.65 -0.52 -0.41 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.41 0.52 0.65 0.81 1 1.28 1.6 1.81
64 -1.63 -1.44 -1.15 -0.9 -0.59 -0.47 -0.37 -0.27 -0.18 -0.09 0 0.09 0.18 0.27 0.37 0.47 0.59 0.73 0.9 1.15 1.44 1.63
96 -1.33 -1.18 -0.94 -0.73 -0.48 -0.39 -0.3 -0.22 -0.15 -0.07 0 0.07 0.15 0.22 0.3 0.39 0.48 0.59 0.73 0.94 1.18 1.33
128 -1.15 -1.02 -0.82 -0.64 -0.42 -0.33 -0.26 -0.19 -0.13 -0.06 0 0.06 0.13 0.19 0.26 0.33 0.42 0.51 0.64 0.82 1.02 1.15
168 -1.01 -0.89 -0.71 -0.55 -0.36 -0.29 -0.23 -0.17 -0.11 -0.05 0 0.05 0.11 0.17 0.23 0.29 0.36 0.45 0.55 0.71 0.89 1.01

[benlessard]

In chess it take 24 games vs players who have a rating to get a rating. IMO 24 boards approx equal a chess game in term of decision and possibility of mistakes however luck vs no luck mean that it should be divided by 2 at least . So ratings will only worth something when you have about 1200 hands vs opps that have ratings.

Since at the beginning nobody have valid ratings IMO around 1800 hands its going to make some sense (if the calculations are well done).

[Trinidad]

HighLow21, on 2013-November-07, 14:34, said:

Rik just to be clear:
- To me what this looks like is 1.1 is the standard error of the average of the IMPs, which would make sense if you were playing 20-40 boards. (As you said, 26 boards.)

Yep.

HighLow21, on 2013-November-07, 14:34, said:

Rik just to be clear:
- And if I multiply this by the square root of 26 I get 5.6, which would be the standard deviation of the IMPs score on any individual board. This is very consistent with my math.

Again: Yep.

HighLow21, on 2013-November-07, 14:34, said:

- Just to be clear -- 1.1 is the standard deviation of the 7 nightly IMPs averages? Can you type in the data so I can see it?

Of course, the standard deviation varied a little bit. I calculated the results for 7 evenings (just because they fit on one web page). You can find the raw data below. As you can see, the amount of tables in play varied.

Rik

1.62   1.71   2.35   1.53   1.54   1.75   2.52
1.28   1.51   1.27   1.31   1.42   1.63   1.46
0.9   1.13   1.21   1.03   1.3   1.44   1.16
0.75   1.19   1.24   0.86   0.91   1.44   0.93
0.74   1.06   0.92   0.79   0.55   0.5   0.87
0.61   0.48   0.53   0.58   0.41   0.45   0.68
0.17   0.31   0.5   0.5   0.4   0.34   0.59
0.17   0.26   -0.15   0.3   0.21   0.23   0.53
0.1   -0.02   -0.2   0.32   0.2   0.23   0.27
0.04   -0.02   -0.28   -0.04   0.18   0.17   0
-0.04   -0.21   -0.45   -0.18   -0.2   0.06   -0.08
-0.07   -0.59   -0.51   -0.53   -0.27   -0.14   -0.15
-0.17   -0.69   -0.6   -0.94   -0.38   -0.18   -0.35
-0.62   -1.14   -0.66   -0.87   -0.61   -0.58   -0.37
-0.71   -1.35   -0.82   -1.22   -1.14   -0.61   -0.99
-1.21   -1.55   -0.85   -1.27   -1.68   -0.64   -1.06
-1.25   -2.15   -0.92   -1.95   -3.04   -0.7   -1.25
-2.44           -1.18                   -0.75   -1.98
                -1.42                   -0.76   -2.97
                                        -0.92
                                        -1.47
                                        -1.49

[HighLow21]

These are the nightly average IMPs at each table yes?

[HighLow21]

OK so there were between 17 and 22 tables each night, and the standard deviation across all tables varied between 0.94 IMPs and 1.28 IMPs. The average standard deviation per night is 1.07 IMPs and the standard deviation of all the results pooled together is 1.05.

Assuming of course that all pairs (teams?) are of equal ability (they aren't) and each hand is independent of one another (they aren't, but they are probably close enough), this would mean that on a per-hand basis, the standard deviation per hand is between 4.80 IMPs and 6.51 IMPs, with the best estimate being 5.36 IMPs. This may be slightly high if we were talking about the expectation for a single partnership, but I can't imagine the haircut would be more than 10% of this.

Besides, my own recorded results show a standard deviation of 5.28 IMPs per hand, which would work out to 1.04 IMPs for my results on a random 26-board night. I'd expect it to be less than this for a partnership with a single individual, and indeed, if I look at my 3 most common partnerships, the standard deviations are 4.36, 5.03, and 5.21. Pooled, I get 5.02.

So there you have it. On a single board, the standard deviation is around 5 IMPs. On 26 boards, this means a standard deviation of around 1 IMP; on 8 it's around 1.8 IMPs, and on 32 it's around 0.9 IMPs.

[helene_t]

HighLow21, on 2013-November-07, 15:00, said:

To go from 26 boards to 168 boards you would have to divide everything in the left-hand column by the square root of 7, which is the square root of the ratio of hands played. The variation decreases as a function of the square root of the number of observations.

Now you are assuming that the variance is entirely random. To see why this can't be correct, imagine that you went from 100,000 boards to a million boards. Surely you wouldn't expect the SD of the mean crossimps per pair to decrease by a factor sqrt(10).

I am afraid there is no simple way out of this. The more the field is dominated by "stupid idiots", the more the random variance will be, but the same is probably true for the skill variance.

[Trinidad]

HighLow21, on 2013-November-07, 18:12, said:

These are the nightly average IMPs at each table yes?

No. Sorry, I should have been clearer.

There are 7 columns. Each column represents one night. In each columns is the final result for each evening: the ranking. So, on the first night, the winning pair had an average of 1.62 IMPs per board, number 2 scored 1.28 IMPs per board, etc.

Every pair played 26 boards, except when there were an odd number of pairs when there was obviously a sit out. In those cases 13 of the pairs played 24 boards only. Obviously, their average scores are the sum of their IMP scores divided by 24.

Rik

[HighLow21]

Trinidad, on 2013-November-08, 03:47, said:

No. Sorry, I should have been clearer.

There are 7 columns. Each column represents one night. In each columns is the final result for each evening: the ranking. So, on the first night, the winning pair had an average of 1.62 IMPs per board, number 2 scored 1.28 IMPs per board, etc.

Every pair played 26 boards, except when there were an odd number of pairs when there was obviously a sit out. In those cases 13 of the pairs played 24 boards only. Obviously, their average scores are the sum of their IMP scores divided by 24.

Rik

Yes that's the way I interpreted it.

[HighLow21]

helene_t, on 2013-November-08, 03:10, said:

Now you are assuming that the variance is entirely random. To see why this can't be correct, imagine that you went from 100,000 boards to a million boards. Surely you wouldn't expect the SD of the mean crossimps per pair to decrease by a factor sqrt(10).

I am afraid there is no simple way out of this. The more the field is dominated by "stupid idiots", the more the random variance will be, but the same is probably true for the skill variance.

No, that's correct -- I was assuming that you were drawing from the same pool of results and that all results were random and independent. It helps to assume all pairs are actually of equal skill in this calculation; clearly this isn't the case, but unless the pool contains pairs that are wildly better or worse than the others, it shouldn't matter terribly much.

And agreed -- the more rabbits/fools in the pool, the more the standard deviation should increase.

The main point of the conversation -- figuring out a rough estimate of how much variation there is, hand-by-hand, seems to stand though. I come to an estimate of 5 IMPs per hand under normal conditions and am fully prepared for the correct answer to be 4 or 6, but not 2 or 10. What do you think?

[HighLow21]

Trinidad, on 2013-November-08, 03:47, said:

No. Sorry, I should have been clearer.

There are 7 columns. Each column represents one night. In each columns is the final result for each evening: the ranking. So, on the first night, the winning pair had an average of 1.62 IMPs per board, number 2 scored 1.28 IMPs per board, etc.

Every pair played 26 boards, except when there were an odd number of pairs when there was obviously a sit out. In those cases 13 of the pairs played 24 boards only. Obviously, their average scores are the sum of their IMP scores divided by 24.

Rik

This was my response to the earlier average of 1.1 IMPs you gave me. The spreadsheet and table I produced, along with discussion, are based on an understanding of your data that matches what you just said.

[Zelandakh]

HighLow21, on 2013-November-09, 11:36, said:

And agreed -- the more rabbits/fools in the pool, the more the standard deviation should increase.

Is a pair playing Precision rabbits or fools? What about Polish Club? Playing a different system from others in the room will increase variation. That does not mean that the standard has been decreased.

[HighLow21]

Zelandakh, on 2013-November-11, 07:14, said:

Is a pair playing Precision rabbits or fools? What about Polish Club? Playing a different system from others in the room will increase variation. That does not mean that the standard has been decreased.

That's true too -- anybody playing an unusual system, particularly one against which the opposing partnership doesn't have agreements, will likely increase the volatility of results.

Basically, the standard deviation is caused by mistakes--anything that increases the frequency or severity of those mistakes (poor players, unusual systems, very wild distributions, etc.) will increase standard deviation. Similarly, a group of expert/world class players would probaby demonstrate a much lower typical swing per board.

[Zelandakh]

No, when different systems are in play you can get swings without any mistakes having been made at all. Even just changing the NT range from 15-17 to 14-16 will create swings without a mistake being required. A swing is not the same as a mistake.

[Trinidad]

It is important to make a distinction between the IMPs per board average for a given pair in a field and the IMPs per board standard deviation on the board set.

When we have a field of pairs, there will be pairs that are stronger than others. There is a range of strengths, and this strength can be quantified, e.g. in the form of an "expected IMPs per board score" for each pair. Each pair has only one given strength, compared to the rest of the field. The whole field is made up of several pairs and this whole field has a "strength distribution". One can model the strength of this field. One can assume, as I did, that the strength of the players is "normally distributed". A normal distribution can be described by two parameters: the average and the standard deviation of the distribution. It is important to realize that this standard deviation is not an error. It is a parameter of the distribution, just like apples on a tree will have a weight distribution. If you weight two apples and you get a different result, it is likely that it is because of the difference in weight of the apples, not because of a random error in the scale.

Unfortunately, errors are also expressed as standard deviations. It is a bit like the difference between pound (mass) and pound (force): They have the same name, and there is a relation between the two, but they are fundamentally different things.

The errors only come into the discussion once we want to measure the strength of a pair, e.g. in a tournament. And we want to measure them, because we don't know the strength of the pairs. We know that each pair must have a strength, but we just don't know its value. So, we estimate their strength, based on the results of the tournament. And when we estimate, we make errors.

In a bridge tournament, there are large random errors in the measurement. This is one of the charms in bridge: There is a realistic probability that Aunt Millie - Uncle Bob beat Meckwell on a single board. It would be premature to conclude that Aunt Millie - Uncle Bob are stronger that Meckwell based on that one board (though I would never deny Aunt Millie and Uncle Bob their moment of glory). The probability that Aunt Millie and Uncle Bob manage that on more boards gets less and less realistic.

So, if we play more and more boards, our tournament result will converge towards the true "strength distribution". The error in the estimate (whether due to the variation in the swinginess of boards or random differences caused by kings sitting over or under aces) will get closer to 0. In contrast, the standard deviation in the pair strength distribution will remain unchanged, because the true value of all the pairs' strength doesnot depend on the measurement.

So, what we are doing in a bridge tournament is the same as determining the weight distribution of the apples on a windy day with a scale that is jumping up and down in the wind: we have a large error in the measurement of the weight of the individual apples. But if we keep repeating the measurement over and over again, we will get a better estimated value for the weight of each apple. And we can use these to determine the weight distribution of the apples in the tree.

In this thread, the question was more or less: the mass of my apple (my bridge strength) from my tree (played in BBO) is x% (x IMPs/bd) larger than the average. What does that mean? I took another tree (bridge club), measured the weight distribution of its apples (measured the strength distribution of the players) and said: "If your tree (BBO) is comparable to mine (my bridge club), it would mean that so many % of the apples (pairs) are lighter (worse) than yours (you)."

All this shows that it is perfectly possible to compare apples and pairs...

Rik

[awm]

It may be important to keep in mind that IMP expectations are not transitive (i.e. if pair A is +1 IMP/bd against B and B is +1 IMP/bd against C it does not follow that A is +2 IMP/bd against C). In fact due to methods and style you cannot cleanly rank pairs like this. Further, it is difficult to compare scores against a field when the field is very far off from the pairs being compared.

[HighLow21]

Zelandakh, on 2013-November-12, 04:27, said:

No, when different systems are in play you can get swings without any mistakes having been made at all. Even just changing the NT range from 15-17 to 14-16 will create swings without a mistake being required. A swing is not the same as a mistake.

It depends on how you define mistake, doesn't it? Any time a pair fails to get its optimal result on the board, it could be argued that it's a mistake. Many such mistakes will be made routinely by even the best players in the world, but if you define it that way, swing = mistake.

[HighLow21]

Trinidad, on 2013-November-12, 07:57, said:

It is important to make a distinction between the IMPs per board average for a given pair in a field and the IMPs per board standard deviation on the board set.

When we have a field of pairs, there will be pairs that are stronger than others. There is a range of strengths, and this strength can be quantified, e.g. in the form of an "expected IMPs per board score" for each pair. Each pair has only one given strength, compared to the rest of the field. The whole field is made up of several pairs and this whole field has a "strength distribution". One can model the strength of this field. One can assume, as I did, that the strength of the players is "normally distributed". A normal distribution can be described by two parameters: the average and the standard deviation of the distribution. It is important to realize that this standard deviation is not an error. It is a parameter of the distribution, just like apples on a tree will have a weight distribution. If you weight two apples and you get a different result, it is likely that it is because of the difference in weight of the apples, not because of a random error in the scale.

Unfortunately, errors are also expressed as standard deviations. It is a bit like the difference between pound (mass) and pound (force): They have the same name, and there is a relation between the two, but they are fundamentally different things.

The errors only come into the discussion once we want to measure the strength of a pair, e.g. in a tournament. And we want to measure them, because we don't know the strength of the pairs. We know that each pair must have a strength, but we just don't know its value. So, we estimate their strength, based on the results of the tournament. And when we estimate, we make errors.

In a bridge tournament, there are large random errors in the measurement. This is one of the charms in bridge: There is a realistic probability that Aunt Millie - Uncle Bob beat Meckwell on a single board. It would be premature to conclude that Aunt Millie - Uncle Bob are stronger that Meckwell based on that one board (though I would never deny Aunt Millie and Uncle Bob their moment of glory). The probability that Aunt Millie and Uncle Bob manage that on more boards gets less and less realistic.

So, if we play more and more boards, our tournament result will converge towards the true "strength distribution". The error in the estimate (whether due to the variation in the swinginess of boards or random differences caused by kings sitting over or under aces) will get closer to 0. In contrast, the standard deviation in the pair strength distribution will remain unchanged, because the true value of all the pairs' strength doesnot depend on the measurement.

So, what we are doing in a bridge tournament is the same as determining the weight distribution of the apples on a windy day with a scale that is jumping up and down in the wind: we have a large error in the measurement of the weight of the individual apples. But if we keep repeating the measurement over and over again, we will get a better estimated value for the weight of each apple. And we can use these to determine the weight distribution of the apples in the tree.

In this thread, the question was more or less: the mass of my apple (my bridge strength) from my tree (played in BBO) is x% (x IMPs/bd) larger than the average. What does that mean? I took another tree (bridge club), measured the weight distribution of its apples (measured the strength distribution of the players) and said: "If your tree (BBO) is comparable to mine (my bridge club), it would mean that so many % of the apples (pairs) are lighter (worse) than yours (you)."

All this shows that it is perfectly possible to compare apples and pairs...

Rik

Good explanation. I think the most important point that follows is that some of the variation you'll see in the pooled results will be caused by randomness, and some will be caused by the relative strength/weakness of individual pairs. The more variation in talent across the pairs, the higher the standard deviation of the observed results. The randomness component will, of course, always be there irrespective of the relative quality of the pairs.