[skrshawk]

I've been somewhat hesitant to bring up this subject on the forums given some of the very heated debates this has caused on the BBO_Community list in the past.  Also, I'm going to be gone for most of the morning my time so I'll have a nice flood of posts waiting for me .

The subject is ratings.

Many people I've talked to are very vocal in expressing their dislike of the concept and I can very much see their point.  Being a below-average player and knowing what it's like to not have the "protection" of being a yellow when it comes to abusive partners and opponents, I'm treading on very thin ice.  As a developer, the abuses to the software and the service when ratings are relied upon are familiar to me and not something I want to face either.

I came up with a "ratings" system on the old BBO_Community list that came up with mixed support, but far stronger than that of a strict performance-based system.  I termed it "user ratings" because of the nature.

User ratings would work like this - each player would have a ratings score of between 1 and 5 in both skill and courtesy.  This is not intended to be a scientific or accurate rating; it is intended to be an opinion-based rating.  The question would be phrased in terms of interpreting the results as "How do other BBO users feel this user rates in skill and courtesy?"

To avoid some of the most common abuses, only one vote could be given for any particular player from a specific IP address every seven days.  To give a rating the player must have sat down and played at least three hands with the user (partner or opponent).  The top and bottom 5% of results would be discarded as soon as the user reaches 20 votes.

As a reward for attaining high ratings (say, 4.5 or higher), users could get different colored stars for their achievement.  Players known for their courtesy could be given red stars, players known for their skill silver stars, and a multicolored star for those rare players able to be known as both.

Another thing - the ratings system can be completely turned off by a particular user if they don't wish to participate.  Alternately, the user may choose to make their ratings private so they can be the only ones to see them (but to qualify for a star ratings must be visible).  Those desiring them to be visible would have them shown in their profile.  The system would be disabled by default.

I'll now let all of you loose on these ideas .

Take care,
John

[bglover]

I see one basic flaw with your idea-- the skill ratings are likely to be of little merit and quite posibly deceiving.

The reason I say this is... when a semi-skilled player sits in with relative novices that player is going to acheive high play ratings when such a rating may not be merited.

If I wanted to raise my skill raiting if I am an average player, what would prevent me from playing with novices who would think I am some sort of bridge god? Nothing that I can see.

I guess there is a counter-argument-- this type of thing would encourage more skilled players to play with novices and that might be a good thing.. But, ultimately, I think that it would hurt the advanced+ or better players' games (people would artificially build up their skill ratings, then try to "move up" to the better games and be out of their element...). IF the skill rating was based on the level of competition-- here comes that "ratings" thing again-- then there would be more validity. But, since that is what this is meant to avoid, that isn't workable to begin with, so the basic flaw I mentioned is inherent here.

[hrothgar]

Quote

To avoid some of the most common abuses, only one vote could be given for any particular player from a specific IP address every seven days.  To give a rating the player must have sat down and played at least three hands with the user (partner or opponent).  The top and bottom 5% of results would be discarded as soon as the user reaches 20 votes.

A few quick comments

First and foremost:

Dynamic IP address assignment is going to cause all sorts of problems if you plan to use IP addresses as a placeholder for user IDs.  If you want to implement such a rating system (and I don't think that you should), you should implement some real security.

Second, I don't see any reason why the top and bottom 5% of votes should be discarded.  If the ratings are bounded on [1, 5] then this won't have a very significant effect on the rating system.

Finally, if ratings can be supressed, then I would expect that the vast majority of people below some level (hypothetically below 3.0) will hide their ranks.  Personally, I wold prefer a system that displays the rankings of the top 30% of players and automatically left the rest blank.  This scheme would give players something achievable to shoot at.  Anyone without a ranking might still feel that they are above averable.

[uday]

I think we should separate skill-ratings from social-ratings.

Maybe we should track and display a "social rating" - based on feedback and comlpaints from other players as well as time spent at BBO.   The software could automatically factor in things like "number of times player left table after auction" or "number of times player booted someone" or "number of hands lpayed on BBO."   Clearly, some details are lacking

Similarly for a skill-rating, although I'm not sure we'll ever want these.   DeanRover suggested a while back that we should compute skill ratings anyway we like, but not display them. Instead, when player A looked at player B's profile, he'd see an indicator of whether A&B were compatible (ie, their skill ratings were close together).  

[inquiry]

For skill ratings, I think what until the tournments are added to the software. Then give something like a batting average in the tournment play. Maybe 1.000  for a win, 0.75 for second, 0.50 for third, 0.25 for 4th, 0.125 for 5th, 0.06 for 6th and 0 for anything else (or any other scale).  Those who win or come in the top three regularly in tournments will have a very high rating, those who never place will be 0 (as will anyone who never plays in one). Say if you play twice and win once and come in last once, your average would be 0.500. If you always place 1, 2 or 3 your average would be higher than 0.50, etc. Not perfect, but at least it is based upon results.

I join the others expressing concern that ratings based upon opinions of others maybe weak. If players can not estimate their own skills accurately (calling themselves experts when they don't even know what a vienna coupe is, or lebehnshol), how in the world can they accurately rate other players who they play a few hands against? However, the courtesy stars and the identificition of players who frequently leave tables after bidding or overly boot players would be more reliable and I could favor that.  

[dageaux]

interesting topic. i like the idea of skill ratings, but i tend to agree with inquiry that a tournament setting should be the method used.

here's an idea (probably impossible to implement, even if it's a good one). skill should be based on result. why not either randomly or with forethought choose 10 hands an hour and designate those hands as 'rated' ones?

the number 10 is arbitrary, it could be any number. randomly choosing them seems best. something like 'hands 1,3,5,7,9 for first the 30 minutes between the hours of 11 am and noon, hands 10, 12, etc for the second 30 minutes'

as an example, by the end of the week i'll have played maybe 200 hands of bridge. of those hands, 20 will have been randomly chosen by the administrators (the same 20 that others are rated by). my scores on those 20 hands would be averaged, etc.

i don't know, what do you think?

[keylime]

A selective average of twenty results isn't statistically solvent because you are not allowing for randomness in the sample. If we went to a snapshot averaging method, you'd need at least 25-30 percent of the total hands playes to start entertaining proper variance of means.