[hrothgar]

mikeh, on Apr 20 2007, 06:32 PM, said:

There is also a luck of the draw issue. Last week, on the Sunday Swiss, my team struggled early but finished strong: going into the last match we had a mathematical shot at winning, and ended 3rd. My strong suspicion is that we would have been demoted on the SOS analysis, because we never faced either of the teams that finished ahead of us.... nor did we play the 4th or 5th place teams.... .

You also need to consider the flip side of the equation:

You placed third in a Swiss Teams event without ever facing the teams that placed first, second, fourth, or fifth. If I placed behind you in that event I'd be bitching about the "Swiss Gambit" which allowed you to place while never facing any strong opposition....

[pclayton]

hrothgar, on Apr 20 2007, 10:50 AM, said:

mikeh, on Apr 20 2007, 06:32 PM, said:

There is also a luck of the draw issue. Last week, on the Sunday Swiss, my team struggled early but finished strong: going into the last match we had a mathematical shot at winning, and ended 3rd. My strong suspicion is that we would have been demoted on the SOS analysis, because we never faced either of the teams that finished ahead of us.... nor did we play the 4th or 5th place teams.... .

You also need to consider the flip side of the equation:

You placed third in a Swiss Teams event without ever facing the teams that placed first, second, fourth, or fifth. If I placed behind you in that event I'd be bitching about the "Swiss Gambit" which allowed you to place while never facing and strong opposition....

Thats exactly the argument the NCAA posed before setting up the BCS.

[hrothgar]

jdonn, on Apr 20 2007, 09:45 PM, said:

It's simple to you. It's simple to me. Do you think it's simple to my grandmother, or even my mother?

If you believe so, I'll let you be the one to try and make her understand why she scored the most victory points and didn't win.

I started playing around with this topic because the Australians were interested in improving the format of their team trials system. Currently, the Aussies hold a VERY big Swiss teams event called the South-West Pacific Teams which has (approximately) 256 teams organized in two pools of 128. This is an event in and of itself.

The top 8 teams from each pool advance into a second event called the NOTs (I think that this stands for the National Open Teams). These 16 teams compete in a Knockout type event to determine the Australian representative for major international events.

When I posted these results the Oz-One forum, I suggested side stepping the entire issue that you raise. Use a traditional Swiss Teams style scoring system for the SWPTs. All the master points and trophies and such that the bunnies care about will use this scoring system.

The Strength of Schedule correction get applied in parallel. It is only used to identify the teams that qualify for the NOTs and to arrive at seeding. In theory, the folks competing in the NOTs are much more interested in the accuracy of the event, possess more technical sophistication, and would be more willing to accept a bit more complexity in the scoring system.

Comment 2:

Your mother and your grandmother accept the VP scales without questioning them. The math underlying Victory Points and IMPs is every bit as complex as what we're talking about here. You mother and grandmother accept this because they've been shielded from its complexity. I certainly agree that change the system would be quite difficult, however, I suspect that this has much more to do with tradition and hidebound thinking than "complexity".

[jdonn]

The imp and victory point scales convert a score to a different scale without changing the relative rank of the scores. It may not be any less complex than what you suggest on the level at which most people think, but at least they know if they scored the "most", it will still be the most all all the scales. Scoring +500 when the other table scores +100 in your seats is "good" and everyone knows it. The fact that some people might not understand how this will directly translate into imps or victory points doesn't matter, since they know they are going to get some amount of those things to reward the good score. If you told them this score would sometimes be worth 10 imps, sometimes 5, sometimes 18, sometimes -2, you don't think they would find this more complex?

You may be right that complexity is the wrong word to use, but I don't think tradition is right either. I think logic is. Which is to say, if you use a format in which a team scores the most victory points and doesn't win, this will go completely against most peoples' basic sense of logic and will leave them unsatisfied.

[mikeh]

awm, on Apr 20 2007, 01:21 PM, said:

As for swiss teams, while I understand Mike's point, wouldn't it be frustrating to be in first place with a round to go, having played virtually all the top teams and won, then play a tight match with the second place team ending in a draw, only to have the third place team pass you because they got an easy opponent in the last round (and never had to play any of the other teams in the top five)? I'd think this also would leave a bitter taste in the mouth of some competitors.

well, yes and no

Of course, seeing the third or fourth place team slide by me in the last match (and it has happened to me) is annoying... but, and this is a very big but.... I had control of the outcome. All I had to do was win, and usually in this situation, not by a huge margin, thanks to the compression of big wins by the VP scale usually in use. But the SoS adjustment is out of my control, and may indeed be ourely random... altho I suppose that, properly implemented, the match pairings should be according to current SoS rankings, not merely current number of VPs.

However, this also gives rise to (likely) perceived fairness issues: if we rank teams not by pure VPs but by SoS rankings, then we may get the seemingly wierd result that the top two teams, in terms of VPs won, NEVER meet... even on the last round. Heck, if I were on one of those, and I saw my counterpart play a weaker team than I did, because I have a good SoS, and score a blitz while I have a tough match, I am surely going to be very annoyed!

At its root is the fact that there is too much randomness inherent in short matches, even with duplicate boards. The current method of matching by VP scores is simple and reasonably useful..... in the context of the events run as Swiss matches.... maybe your Australian event would be an exception to this proposition, but even there, so long as the swiss is merely a precursor to more stringent metrics of ability, then the good teams will usually make whatever the cutoff is for the next stage.

In the meantime, for the 'bridge for the masses' fields for which Swiss was adopted, my advice is don't fix what ain't broken. The masses like and understand VP swiss without SoS metrics... and never, ever put in place a scoring system that can be applied only by a computer utilizing data unknown to the players.

[awm]

The problem though is with the "leaderboard" reflecting something different than what's actually happening. Sure, if there's a list of teams with point totals somewhere and my team has the most points, but didn't win, I can see not liking what's going on.

But it's easy enough to do something like adding points for playing teams in the top part of the standings so the scoreboard reflects who's actually winning.

In fact it does seem strange and vaguely frustrating when my team's doing pretty well and draws the Meckstroth-Rodwell-Soloway-client team, holds our own against them but loses by a small margin, and then finds that a dozen teams have passed us by beating up on weak opponents.

It's always seemed strange to me that in swiss scoring, if the teams are roughly ranked "correctly" then pretty much every team expects a draw. So subsequent rounds wouldn't seem to widen the gaps between the better teams and the worse teams, since the better teams have better opponents. I suppose in some ways it's nice that it's very hard for a team to separate itself from the pack, but this leaves open the possibility of a team getting a "lucky draw" in a late round where they have lots of swingy boards against bad opponents, and passing everyone without really beating the good teams.

[mycroft]

The big reason that SoS works in the BCS ranking system (to the extent that it does work, of course - what a *stupid* way to run a National Championship, it rivals that of several NCBOs!) is that the teams set their schedules. Sure, this might be the year that Auburn flames out, and there's always the cross-state rival game for Homecoming that we have to put in there no matter how much it hurts our schedule because we make half our money on that one game, but generally, the teams have a good guess what their SoS is going to be going in, and whether they need to push for 11-0 or if 10-1 is going to get them to a BCS spot (or, if they're weaker, whether their SoS and a 6-5 record will make it to the Joe's Garage Durian Bowl, sponsored by Mom's Pizza Joint (still on ABC, for some reason, however)).

Give me a Bridge tournament, where I know most of the teams, and I am allowed to choose my opponents (with their permission, of course), then SoS is reasonable. Yeah, there are going to be several teams going 8-0, because they've picked the fish who want to play up; but their SoS is going to be horrible. If I can go 7-1 playing mostly good teams, I'll do better (as I should).

But as Swiss is a lottery, and all SoS is doing is reversing the lottery dynamics.
Round 1: paired with the worst team in the room. Win 20-0.
Round 2: paired with a big winner of rabbit vs ferret. Win 20-0.
Round 3: caught a decent team, but they blew away their first set of fish and got lucky on the second round in a swingy match. Win 16-4.

I'm never going to win this, am I? I'm already so far in the hole on SoS that I'll never recover, and my VP total is going to mean that I get paired with all the other contenders all night, making it very hard to win *and* pick up the SoS.

If you're looking to pick 8 from 128, you're always going to have an issue, unless you can play for a week - 2 from 32 could probably be done RR over 3-4 days. Frankly, that's the problem that the BCS has - they have to find the best 2 pairs of 2 teams from a class of 100+, based on 11 one-on-one matches. *Nothing* is going to get that unarguably right. At least in the NCAA, they have to find 64 (yeah, I know - but the play-in game is stupid and doesn't count) out of the 200 or so, based on 30 or so games.

Michael.

[cherdano]

If you are using VP + some fraction of SoS to determine the winner, then I think it is understood that you use the same for the leaderboard between rounds, and to determine the pairings.

[barmar]

jdonn, on Apr 20 2007, 11:49 AM, said:

I don't understand the desire to achieve this goal of accuracy. If all we want is for the best team to win as often as possible lets not even play, lets just take a vote on who is the best team and give them some masterpoints. And then why would anyone but the best team even show up?

If you never play, then how would anyone know who the best team is? Who would you vote for? It's not a beauty pageant

Furthermore, the "best" changes over time. New players come onto the scene, old players retire, partnerships and teams change, etc. Richard's simulation simply assigned rankings to all its teams, and from this it presumably derived probabilities about the results of matches between any pair of teams. But in the real world we don't know the current rankings until AFTER everyone plays. As they say about the lottery, "You have to be in it to win it."

So the point of the exercise is that since we're going to hold tournaments, we'd like the results to be meaningful -- it should be more likely that the better teams win. So the scoring system and tournament format should reflect this expectation.

But as others have pointed out, there may be conflicting goals, such as simplicity of scoring and finishing the tournament in a reasonable amount of time. 9 rounds of 20 boards would require 3 days to play out -- not your typical Sunday Swiss.

[jdonn]

barmar, on Apr 21 2007, 10:51 PM, said:

So the point of the exercise is that since we're going to hold tournaments, we'd like the results to be meaningful -- it should be more likely that the better teams win.

Do you mean more likely that the better team wins than that the better team loses? We already have that.

Or do you mean more likely that the better team wins under a new system than that the better team wins under the current system? I strongly disagree with this, it is at best a debatable assertion, on which I am still firmly on the other side.

This seems like an exercise to make it as close to a certainty as possible that the better team always wins any time they play. Why!!! It is the chance of upsets by weaker teams (while still losing more often than winning) that make things fun and interesting. The results are still meaningful when taken over any sort of long term. When taken for an individual day or event I can't fathom why someone would want the best team to always win or anything close to it.

Adam made an earlier post sort of implying that he thought I felt this way for less meaningful events, but for really "important" stuff something more accurate would be better. No no no, you have to give all the teams something meaningful to play for. If the only goal was to pick the best team every time, a vote among the players would be far more accurate than actually playing. Or do you just quit when you sit down against Meckwell?

[cherdano]

So Josh, why do the USBF trials use seeding, and take this long? They could as well play a 3-day round-robin.

I think for events such as trials, pretty much everyone would disagree with your point. And in bridge there is always enough luck that the better team won't always win.

[jdonn]

cherdano, on Apr 22 2007, 02:08 AM, said:

So Josh, why do the USBF trials use seeding, and take this long? They could as well play a 3-day round-robin.

So why do they play at all? They could just send the top seed.

Quote

I think for events such as trials, pretty much everyone would disagree with your point. And in bridge there is always enough luck that the better team won't always win.

It seems that would not be true if lots of people here had their way.

[mike777]

I agree.

Why waste all this time and money of playing...cannot we have the Old wiseman just pick the best players...We do want to win number one...yes?...

Imp pairs/Butler pairs..whatever.......round robin pairs..or this expert seeding ..just seems a waste of time.....why let lucky players go forward?

[hrothgar]

jdonn, on Apr 22 2007, 09:41 AM, said:

Or do you mean more likely that the better team wins under a new system than that the better team wins under the current system? I strongly disagree with this, it is at best a debatable assertion, on which I am still firmly on the other side.

One of the advantages of the methodology that we are using is that it provides an objective mechanism to test these types of assertions. Simply put, the numbers don't agree with you.

I don't have any problem with discussing this topic (Its part of the reason that I posted these results). However, it would be useful if people critical of the results were able to provide a more useful critique. There are three (broad) areas in which these results can be criticized:

1. The model that Alex chose does not accurate describe results at a bridge table. (For example, one could claim that there is autocorrelation between board results or what have you)

2. The model may be correct, however, Alex chose the wrong set of operating parameters. Hypothetically, one might claim that the strength of bridge players is better described by a uniform distribution than a normal distribution.

3. Implementation problems: Even if you find a better way to skin the cat, no one would ever use it

[hrothgar]

mike777, on Apr 22 2007, 11:28 AM, said:

I agree.

Why waste all this time and money of playing...cannot we have the Old wiseman just pick the best players...We do want to win number one...yes?...

Imp pairs/Butler pairs..whatever.......round robin pairs..or this expert seeding ..just seems a waste of time.....why let lucky players go forward?

There are a number of countries that use this type of selection process for their National teams. For example, I believe that this is used in Italy. I think that at least one of the Scandinavian countries does as well.

Obviously, the selectors need some kind of data that they can use to make their decisions which pretty much needs to come from board results. As I understand matters, the primary critique of the selector type systems boil down to

1. It requires a good selector (Or at the very least a very good process that the selectors can apply)

2. It destroys any chance that sponsors make the team

[Free]

Richard, I was wondering if it would be better to multiply results with the number of VP your opponents got, instead of just adding them.

For example (short one):
round 1 25-1 against losers
round 2 17-13 against good guys
round 3 20-10 against reasonable players

Total of 3 rounds is 62
Suppose in the end our good guys have 60VP, reasonable players have 40VP, the losers have 15VP.
Our total would become 25*15 + 17*60 + 20*40

Dunno if this would be an even better representation of your performance on that tournament, but intuitively it feels better...

[hrothgar]

Free, on Apr 22 2007, 05:58 PM, said:

Richard, I was wondering if it would be better to multiply results with the number of VP your opponents got, instead of just adding them.

I proposed a very similar adjustment based on the fraction of the available VPs that you scored against a given team. (For example, if you scored a 10-10 tie against team i, you'd get 50% of team i's total VPs). I agree that intuitively this makes a lot of sense. (At the same time, I'm not sure how good my intuition is on these problems) Unfortunately, we haven't had the chance to test this yet and see whether it improves the accuracy of the SoS adjustment.

Right now, we think that "naive" SoS adjustment has a significant enough impact on the accuracy of the tournament that its worth presenting the basic results and solicit comments.

Over time, we hope that we can do some further work designed to improve the accuracy of the SoS metric. In theory, we could adopt something significantly more complex like some of the structures that Gerben was using to evaluate incomplete tournaments.

[jdonn]

hrothgar, on Apr 22 2007, 06:55 AM, said:

jdonn, on Apr 22 2007, 09:41 AM, said:

Or do you mean more likely that the better team wins under a new system than that the better team wins under the current system? I strongly disagree with this, it is at best a debatable assertion, on which I am still firmly on the other side.

One of the advantages of the methodology that we are using is that it provides an objective mechanism to test these types of assertions. Simply put, the numbers don't agree with you.

I don't have any problem with discussing this topic (Its part of the reason that I posted these results). However, it would be useful if people critical of the results were able to provide a more useful critique. There are three (broad) areas in which these results can be criticized:

1. The model that Alex chose does not accurate describe results at a bridge table. (For example, one could claim that there is autocorrelation between board results or what have you)

2. The model may be correct, however, Alex chose the wrong set of operating parameters. Hypothetically, one might claim that the strength of bridge players is better described by a uniform distribution than a normal distribution.

3. Implementation problems: Even if you find a better way to skin the cat, no one would ever use it

How can numbers disagree with a subjective opinion? I'll try again.

I do not think making the results of swiss team events more "accurate" is a good thing. Most people seem to be taking for granted that it is. I am not criticizing the effectiveness or accuracy of the model itself, I'm saying I don't ever want to see it implemented (which thankfully, I doubt it will be.)

[Ant590]

hrothgar, on Apr 22 2007, 10:15 AM, said:

I proposed a very similar adjustment  based on the fraction of the available VPs that you scored against a given team.  (For example, if you scored a 10-10 tie against team i, you'd get 50% of team i's total VPs).  I agree that intuitively this makes a lot of sense.  (At the same time, I'm not sure how good my intuition is on these problems)

I don't trust my intuition also, but I feel that this method would make in very hard for teams to climb and fall in the rankings in the latter stages of events assuming 11/9 10/8s etc are the most common scores.

[Free]

Isn't that the point?