[mikeh]

whereagles, on Dec 13 2007, 05:08 PM, said:

mikeh, on Dec 13 2007, 09:54 PM, said:

my take on the idea that an expert makes an average of 1 error per 20 boards says a lot more about the level of expertise of the poster than about the actual error rate of an expert

You know why you'll never be a good player? You waste too much time bickering with others instead of improving your own game. I, on the other hand, won't bother with you. Have a nice day.

Thanks for the tip

[hotShot]

mike777, on Dec 13 2007, 11:02 PM, said:

How does your program define an error and why is that the correct definition compared to other commonly accepted ones?


Again if we cannot even agree what an error is why bother?
Are all errors weighted equally, why?
Can bidding or lack of bidding induce cardplay error? If so how do you define it and weight it?

These are indeed interesting questions, but I look at it from a practical point of view.

I have a list of about 250 player from which I watched or played more than 40 boards, I find that world champions and our national champions are at the upper end of this list, while BIL member find their place at the lower end of he list.

If I'm interested in a players true rating, I can get me a bunch of boards from myhands and calculate the average error rate. Thats good enough for me.

[awm]

The error rate idea does work a heck of a lot better if you restrict to declarer play. Now things like signaling partner are removed from the equation. You still get a bit of an effect from the opponents (good opponents will find mandatory falsecards and such that make double-dummy play more difficult for example). And of course there is the effect that the right percentage play isn't always the right double-dummy play. But the latter effect should tend to cancel out over a very large number of boards. Obviously it's not right to call this really "error rate" because someone who plays every hand single-dummy perfect will not have a perfect "zero error rate" here. But you'd expect that good players generally take higher percentage lines and therefore produce fewer non-double dummy plays over the long term.

[mike777]

Well no rating is perfect but why not if you win:
1) open wc you are rated wc player
2) open nat you are nat rated player
3) open reg, you are reg rated player
4) etc, etc

yes this is not perfect but it seems faster, simpler, and you do not need a computer and good enough ?

I assume the goal is to have a "good enough" rating system for some unknown reason.

[jdonn]

mike777, on Dec 13 2007, 07:07 PM, said:

Well no rating is perfect but why not if you win:
1) open wc you are rated wc player
2) open nat you are nat rated player
3) open reg, you are reg rated player
4) etc, etc

yes this is not perfect but it seems faster, simpler, and you do not need a computer and good enough ?

So you think Zia is not world class?

[mike777]

jdonn, on Dec 13 2007, 07:09 PM, said:

mike777, on Dec 13 2007, 07:07 PM, said:

Well no rating is perfect but why not if you win:
1) open wc you are rated wc player
2) open nat you are nat rated player
3) open reg, you are reg rated player
4) etc, etc

yes this is not perfect but it seems faster, simpler, and you do not need a computer and good enough ?

So you think Zia is not world class?

Seems the rating sytem is good enough for bbo if that is the goal. I did say not perfect, just good enough, fast and simple. If you need a computer to tell you what Zia is, fair enough.

Keep in mind the goal in Not an absolute accurate rating, is seems to be some unknown goal...I just think this is good enough for that goal.

[jdonn]

mike777, on Dec 13 2007, 07:12 PM, said:

Seems the rating sytem is good enough for bbo if that is the goal. I did say not perfect, just good enough, fast and simple. If you need a computer to tell you what Zia is, fair enough.

It's fast and simple. But I neither know what you mean by 'good enough', nor agree that it is.

[mike777]

jdonn, on Dec 13 2007, 07:16 PM, said:

mike777, on Dec 13 2007, 07:12 PM, said:

Seems the rating sytem is good enough for bbo if that is the goal. I did say not perfect, just good enough, fast and simple. If you need a computer to tell you what Zia is, fair enough.

It's fast and simple. But I neither know what you mean by 'good enough', nor agree that it is.

Well since you have not stated what the goal is, how you expect to measure the success of that goal and why that goal is important, what can I say.

Seems better than anything else I have read so far but if you got a better plan or goal, ok.

[pclayton]

awm, on Dec 13 2007, 11:55 AM, said:

A lot of the systems proposed depend too much on what people "have done." My concern is situations like this:

(1) Player A lives in a small country with few bridge players. He made his country's national junior team when he was in his twenties, basically because he was under 26 and could follow suit. They proceeded to get thrashed in the junior championships. Player A has never done much in open events. Player B lives in a large country with many bridge players. He didn't really get involved with bridge until he was in college, leaving him too far behind to make his national junior team. He has a full-time job (not bridge), but has played in a dozen national-level events in his country, finishing in the top ten several times but never winning. Who's likely to be a better player, A or B? Guess who gets a star on BBO and is rated as "world class" by most people's rating systems? Who might not make "expert" by Frances' system? Hmm....

(2) Player A has been playing bridge since he was in college, and has been retired for the last ten years. He's played in over a hundred national events in his country, with one win and a half dozen other top ten finishes. Player B has been playing bridge for five years while working full time. He's played in ten national events in his country, with no wins but four top ten finishes and always in the top half of the field. Who's likely to be a better player, A or B? Guess who rates as "expert" according to most people's systems? Guess who gets a star?

(3) Player A is a billionaire. He has played in a half-dozen top-flight national events, always on a team of six with the five best players his money can buy. They managed to make the finals of a major event once, but never won. Player B is a graduate student. He has played in a half-dozen top-flight national events, always on a team of six with five of his buddies from college. They managed to make the finals of a major event once, but never won. Who's probably a better player?

Tricky, isn't it?

This gets my vote for post of the month. It really hits home.

[Echognome]

I pretty much agree with everything Adam has posted. I'm glad the juniors get recognized and that the top female players get recognized. I don't see any problem with divvying out accolades. It's an impossible problem from BBO's perspective.

But I do approve of Adam's wording for the guidelines.

[mike777]

The good thing of a worldwide bridge hall of fame is that it acknowledges great players who never won a WC, let alone an open WC. Granted that player may have to wait until what 60 or so to be allowed in and get his or her true rating.

This lets clients as Mrs. Meltzer be rated a WC player and others be rated one step higher, Hall of Famer.

Are the ratings perfect no, but seem good enough.
It just seems a bit unfair to win a few open WC events and not be allowed to call oneself WC on BBO no matter how many cardplay errors the computer says you make.

[mike777]

I guess I just do not understand what the big deal is.

So what if a Nat. champion from tiny bridge country is rated ahead of a Reg rated player from the huge bridge country. Is this really that big of an issue?

[awm]

We can always ask the question: "Why does a rating system matter at all? Who cares?"

I think the success of BBO so far indicates that a rating system isn't all that important. We can survive without one, or with a rather inaccurate one. It's not a hugely important issue.

However, ratings do matter a little bit. Some reasons:

(1) A lot of people would like to see a good rating for themselves, to get an objective measure of how they are playing. Results on boards only go so far, since your results on boards depend an awful lot on who you play with and against.

(2) When one is picking potential partners, opponents, and teammates it's good to have some idea of how good people are. Obviously when dealing with people we compete against on a daily or weekly basis, we have some opinion on who's good and who isn't. But these opinions are often very subjective and inaccurate. And they're likely to be even worse on BBO when we frequently play with or against people we've never met face to face. Anyways, the point is that there's some desire to play with/against people of your approximate level, rather than wasting your time playing against people who are much worse (or much better).

(3) Some people like to kibitz. It can be fun kibitzing friends, but a lot of time the kibitzers want to see the best game possible. With this in mind, it's nice to be able to figure out who the really good players are so we can watch them.

Of course, there are also some negatives to having a rating system. One is that some people are getting worse and would rather not be informed of it (usually age-related issues) or that some people like to believe they are a lot better than they are and ratings would disabuse them of this notion. Another is that having a rating system leads people to care a lot more about their results (i.e. a lot of people would like to have a rating that they feel reflects their skills). This makes it less appealing to play late at night when tired (or drunk) for fear that one's rating will go down. It may create an incentive to cheat. If the rating system is not very accurate, it can also create an incentive not to partner weak players or oppose strong players because this is likely to reduce one's rating (this was a problem with the old OKB Lehman system).

In any case, there are several possible approaches to ratings:

(1) Do away with ratings entirely. Not too many people want this though, because even simple things like picking who to kibitz become tough.

(2) Use self-ratings without any policing of the ratings. This is basically what BBO does now. Some guidelines for how to self-rate (to maintain at least some modicum of accuracy) are probably a good thing. Of course, this leads to constant complaints about people who overrate (or underrate) themselves, and people harping about "I played with an EXPERT and he could hardly follow suit."

(3) Base ratings completely on tournament "wins." BBO does a bit of this, granting stars to people who have represented their country internationally or have won a national event. The problems are set out in my previous post -- like masterpoints, this kind of rating favors participation over skill in many cases. Older people who've played for decades find it easier to accumulate a national win or two simply because they've had more chances. Young people who can make junior teams in a country with few young players also have an advantage (they can "represent their country" without having to make a top-notch Bermuda Bowl team). There's also the problem of wealthy sponsors who may be poor players but hire five elite professionals and win some big event.

(4) Use self-ratings, combined with some form of subjective ratings by partners and opponents. This way, if someone self-rates in a ridiculous way, other people will hopefully tag them and their rating will come back to the consensus of the community. This is the kind of rating system used by a lot of other online services (i.e. Netflix, Amazon). One nice thing about this is that no one has designed a fool-proof objective rating system for bridge yet, and it removes the complaint that the numerical system is poor or inaccurate.

(5) Use some objective system, but make the result only visible to the person being rated. This prevents some of the social "abuses" where people won't play with other people because their rating is too low. But it also prevents some of the comparable social benefits.

(6) Use some objective system, make the result globally viewable. This is basically the OKB approach. It had a lot of negative social effects (one can argue that it would've worked better if the rating system were more accurate -- designing a good rating system for a partnership game like bridge is a tough mathematical problem which hasn't yet been adequately solved). Fred has decided not to do this on BBO, I think for good reason.

[han]

pclayton, on Dec 13 2007, 07:56 PM, said:

awm, on Dec 13 2007, 11:55 AM, said:

A lot of the systems proposed depend too much on what people "have done." My concern is situations like this:

(1) Player A lives in a small country with few bridge players. He made his country's national junior team when he was in his twenties, basically because he was under 26 and could follow suit. They proceeded to get thrashed in the junior championships. Player A has never done much in open events. Player B lives in a large country with many bridge players. He didn't really get involved with bridge until he was in college, leaving him too far behind to make his national junior team. He has a full-time job (not bridge), but has played in a dozen national-level events in his country, finishing in the top ten several times but never winning. Who's likely to be a better player, A or B? Guess who gets a star on BBO and is rated as "world class" by most people's rating systems? Who might not make "expert" by Frances' system? Hmm....

(2) Player A has been playing bridge since he was in college, and has been retired for the last ten years. He's played in over a hundred national events in his country, with one win and a half dozen other top ten finishes. Player B has been playing bridge for five years while working full time. He's played in ten national events in his country, with no wins but four top ten finishes and always in the top half of the field. Who's likely to be a better player, A or B? Guess who rates as "expert" according to most people's systems? Guess who gets a star?

(3) Player A is a billionaire. He has played in a half-dozen top-flight national events, always on a team of six with the five best players his money can buy. They managed to make the finals of a major event once, but never won. Player B is a graduate student. He has played in a half-dozen top-flight national events, always on a team of six with five of his buddies from college. They managed to make the finals of a major event once, but never won. Who's probably a better player?

Tricky, isn't it?

This gets my vote for post of the month. It really hits home.

You and Adam get to keep your expert status, not to worry.

[pclayton]

Hannie, on Dec 13 2007, 06:22 PM, said:

pclayton, on Dec 13 2007, 07:56 PM, said:

awm, on Dec 13 2007, 11:55 AM, said:

A lot of the systems proposed depend too much on what people "have done." My concern is situations like this:

(1) Player A lives in a small country with few bridge players. He made his country's national junior team when he was in his twenties, basically because he was under 26 and could follow suit. They proceeded to get thrashed in the junior championships. Player A has never done much in open events. Player B lives in a large country with many bridge players. He didn't really get involved with bridge until he was in college, leaving him too far behind to make his national junior team. He has a full-time job (not bridge), but has played in a dozen national-level events in his country, finishing in the top ten several times but never winning. Who's likely to be a better player, A or B? Guess who gets a star on BBO and is rated as "world class" by most people's rating systems? Who might not make "expert" by Frances' system? Hmm....

(2) Player A has been playing bridge since he was in college, and has been retired for the last ten years. He's played in over a hundred national events in his country, with one win and a half dozen other top ten finishes. Player B has been playing bridge for five years while working full time. He's played in ten national events in his country, with no wins but four top ten finishes and always in the top half of the field. Who's likely to be a better player, A or B? Guess who rates as "expert" according to most people's systems? Guess who gets a star?

(3) Player A is a billionaire. He has played in a half-dozen top-flight national events, always on a team of six with the five best players his money can buy. They managed to make the finals of a major event once, but never won. Player B is a graduate student. He has played in a half-dozen top-flight national events, always on a team of six with five of his buddies from college. They managed to make the finals of a major event once, but never won. Who's probably a better player?

Tricky, isn't it?

This gets my vote for post of the month. It really hits home.

You and Adam get to keep your expert status, not to worry.

Whew. I kept scrolling down for pigeon and couldn't find it

[uday]

Quote

Use self-ratings, combined with some form of subjective ratings by partners and opponents.

Is the thinking that you'd accumulate subjective ratings w/o worrying about the source of the rating (ie., the rating of the person making the rating ) ?

How could we express the result simply?


For example: 10 people mark XYZ as an expert, 100 as advanced, 200 as intermediate. Now someone checks up on XYZ. Would you expect him to see

expert: 10
adv: 100
int: 200

and draw his own conclusions?

[awm]

uday, on Dec 13 2007, 11:55 PM, said:

Quote

Use self-ratings, combined with some form of subjective ratings by partners and opponents.

Is the thinking that you'd accumulate subjective ratings w/o worrying about the source of the rating (ie., the rating of the person making the rating ) ?

How could we express the result simply?


For example: 10 people mark XYZ as an expert, 100 as advanced, 200 as intermediate. Now someone checks up on XYZ. Would you expect him to see

expert: 10
adv: 100
int: 200

and draw his own conclusions?

There are established ways to do this sort of thing, via reputation systems and so forth. It's a complicated topic so I won't go into a lot of details here (nor do I have more than a basic familiarity with the relevant literature).

I believe Dr Todd implemented such a thing for BBO a while back; you might want to contact him if you're interested in this stuff (or talk to someone at netflix.com, amazon.com, ebay.com, etc).

[hotShot]

Suppose I take a set of boards played by top-WC players.
I give them to a bunch of beginners.
After that I analyze the boards where the suit played is the same.
Suppose I have 20 boards to compare.
The WC-Player made not a single mistake.
Pair A made a trick loosing error in 19 of 20 boards, pair B made a trick loosing error in every board. When they play pair B gives away an extra trick each board, but pair A gives it back 19 of 20 times.

So pair A has 19 times the same result as the WC players and one score is better. Pair B has 19 times the same result as the WC players and one that is worse.

Looking at the results only obviously pair A is of galactic strength worthy to represent earth in intergalactic championships, while pair B is a little weaker than WC probably expert level.

(Self)ranking is hard ......

[sceptic]

actually E Bay have a ratings system. why can Bridgebase not adopt such a system, it would not be hard to work out a method of rating someone and also the ratings have some bearing as to the strength of the person ranking them also, so the higher standard someone is, the more emphasis put on their vote. it would probably take a few months for the rankings to settle down to mean anything

it may be possible to only see the rankings your friends or partners have given to a specific player

[mike777]

So far I have not seen anything easier, that works than what I suggested ages ago.

The goal is not to make Zia the best rated player on bbo.

The goal is not to rate the best double dummy players.

If you win a Nat open tourney or an open WC tourney you are good player on bbo.

If you win a Reg open tourney player you better than most.

that is good enough for bbo.....

We are not making a grandmaster chess list.