BBO Discussion Forums: Tools/advice for doing statistical research? - BBO Discussion Forums

Jump to content

Page 1 of 1
  • You cannot start a new topic
  • You cannot reply to this topic

Tools/advice for doing statistical research?

#1 User is offline   MaxHayden 

  • PipPip
  • Group: Members
  • Posts: 34
  • Joined: 2019-August-25

Posted 2020-May-20, 15:47

Most of the bridge statistics I've seen are statistics about all hands or all situations. But I want to understand how these statistics change depending on the the exact situation you are faced with.

I'm honestly surprised that someone hasn't already come up with a tool that can answer these types of questions. Is there relevant research that I missed? Or tools that I failed to find?

Either way, if anyone knows has suggestions on tools or methods I should be using, I'd appreciate any advice, especially advice that can save me time or hassle.

So far, I've found DDS, which seems to be the best option for a double-dummy solver. But it seems like I'll have to write my own scripts to generate a database of hands or otherwise use the program to create the data I need for any given statistical analysis. My understanding is that there's no consensus on what makes for a quality single-dummy solver. So my current plan is to average over the double-dummy results of a number of hands that share the relevant cards but are otherwise randomized.

To put things mathematically, I want to understand how many "bits" of information it costs to get a certain amount of variance reduction, and how the relevant information and its value changes in different bidding or card play scenarios. My initial plan is to use general linear regression models, but if someone has an alternative approach that would work better, please let me know.

===

Bidding examples:

A ton of people have done hand evaluation analysis, but they mostly focused on how many tricks you can take as declarer. Unfortunately, that's not really what people need to know.

If you could make a system that always gave you the right bid to maximize the score differential, it wouldn't matter that you couldn't tell if 4 was a good sacrifice or a solid game until you saw dummy's cards. Knowing which one you have is information but not relevant information. The difference between what you can take on offense and what you can take on defense usually matters more (per the Law of Total Tricks).

Even if you had a perfect hand evaluation formula, the information you both need is not the same as the information you need. Getting partner exactly what they need to make a decision is the basis of most conventions and almost all of modern competitive bidding. It's also the reason for the captaincy principle and the distinction between telling and asking.

And hand evaluation formulas only report averages over all possible hands. But once bidding has started, you should only average over the remaining possibilities. Different things matter different situations, and the cost of communicating them depends on what has already happened in the bidding. Some information is almost always needed. But if you are designing a convention or a bidding system, you want to know which hands require accounting for something that only matters once in 100 hands. (Because it might matter in 50% of the hands being handled by that convention at that point in the bidding.)

Moreover, your hand's points, however you count them, aren't the only thing that matters. Once you see your cards, you *also* have a prediction about the cards of the other three players. And you should take this into account as you bid and revalue your hand. Knowing that your opponents didn't preempt, overcall, or double reduces the number of possibilities too.

===

Card play examples:

I'd like to create some more detailed probability tables than the ones available in places like the Bridge Encyclopedia.

How often do lines of play or outcomes swing drastically based on additional information that could be revealed in bidding or learned from counting and discards? (And similarly, using game theory, when is honesty optimal vs when does false-carding or keeping quiet during bidding outweigh the benefits of giving partner a correct signal or lead suggestion?)

How often do different card-play techniques and situations end up mattering? How often can you combine lines of play vs having to make a choice?

How does the variance on different lines of play change in light of other information you have? Even if it is still a 50% play, it can be more or less risky in different situations. So whether or not it is worth taking that risk depends on the score you have and the number of boards you have left.

We have rules for how risky you should be on the average board, but can we quantify how you should shade that given the tournament situation and how that situation changes the balance between luck vs skill? If I have X deals left, what should I expect the distribution of hands to look like and how much uncertainty is there?

(To answer these questions, you'll need to know things like, if we have game, how often does the other side have game? When the other side has game, how often do we have a profitable sacrifice? What is the distribution of optimal contracts by score? How often can everyone make 1NT vs someone being able to make 2 of a major or 3 of a minor?)

[Edits made to correct typos]

This post has been edited by MaxHayden: 2020-May-21, 09:16

0

#2 User is offline   hrothgar 

  • PipPipPipPipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 15,372
  • Joined: 2003-February-13
  • Gender:Male
  • Location:Natick, MA
  • Interests:Travel
    Cooking
    Brewing
    Hiking

Posted 2020-May-20, 16:05

With respect to hand generators / double dummy solvers I think that your best best is to use either

1. Thomas Andrew's Deal
2. Hans van Stavern's Dealer

Both are very good choices, however, Deal has better statistical properties. (Deal is capable of dealing all possible bridge hands. Dealer is not. And, while I don't have any evidence to suggest that Dealer produces biased samples, it can't be ruled out)

However, at the end of the day, your choice is more likely to hinge on what language you're more comfortable with.

Dealer using TCL as a scripting language
Deal also uses TCL, but the scripting system is much more reminiscent of C
Alderaan delenda est
1

#3 User is offline   helene_t 

  • The Abbess
  • PipPipPipPipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 17,068
  • Joined: 2004-April-22
  • Gender:Female
  • Location:UK

Posted 2020-May-20, 16:13

Even the Bird&Anthias book used double dummy simulations, so presumably single dummy is just too difficult to be worth it for most purposes:
- It's a PITA to implement as no SD solver is available which can easily be used with Dealer scripts
- For some purposes, it arguably doesn't matter since the "random" deviations between SD and DD won't matter much
- DD is at least a well-defined, reproducible concept. Once we start modelling SD, we can argue about what SD really means from now on to doomsday
- It's slow. You're probably happy to tolerate some small DD-related biases in exchange for 100x larger sample size for a given amount of computer resources

That said, I have seen the following done occasionally:
- SD opening leads combined with DD play thereafter. This might mimmick declarer's advantage for game and partscore hands, and gives some credit to auctions that conceal declarer's hand. Beware, though, that DD already biases slam hands in declarer's favouor, so taking this approach makes it worse. You will get too many thin slams making.
- Calibrating DD to SD using the Vugraph archieve or BridgeBrowser. This is not a trivial task since you have to make a sensible model for the DD-SD deviations. Just a single fixed effect is not so useful.
- Hacking GIB or WB5. If you want to use Jack you will probably have to do it semi-manually.
- Use real data (for example BridgeBrowser). Obviously that is somewhat limited since you can't easily figure out what the bids are supposed to mean (comparing efficiency of different response schemes to 1 is a bit pointless when you don't know who had what negative inference. And some will be playing Precision or WJ).
The world would be such a happy place, if only everyone played Acol :) --- TramTicket
0

#4 User is offline   hrothgar 

  • PipPipPipPipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 15,372
  • Joined: 2003-February-13
  • Gender:Male
  • Location:Natick, MA
  • Interests:Travel
    Cooking
    Brewing
    Hiking

Posted 2020-May-20, 16:13

I think that the project that you are suggesting is incredibly ambitious.

If I were going to kick off this sort of approach, I suspect that I would focus on trying to train a Deep Learning program to play bridge and then give it a couple weeks to really learn how to play bridge rather than approach this analytically.

If you do want to solve this for yourself, I'd recommend focusing on a very narrowly constrained problem and see whether you're able to come up with a satisfactory solution. I personally would suggest trying to compare the efficiency of various auction termination mechanisms following shape resolution in game forcing relay auctions.

If you can come up with really convincing results to this (relatively) simple problem you will have accomplished something significant (and given you a strong foundation to consider other more complicated questions)

And, if you can't solve this (relatively) simple problem then you might need to rethink your approach
Alderaan delenda est
1

#5 User is offline   helene_t 

  • The Abbess
  • PipPipPipPipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 17,068
  • Joined: 2004-April-22
  • Gender:Female
  • Location:UK

Posted 2020-May-20, 16:17

View Posthrothgar, on 2020-May-20, 16:05, said:

Dealer using TCL as a scripting language
Dealer also uses TCL, but the scripting system is much more reminiscent of C

The 1st should be Deal, not Dealer, I think?
The world would be such a happy place, if only everyone played Acol :) --- TramTicket
0

#6 User is offline   hrothgar 

  • PipPipPipPipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 15,372
  • Joined: 2003-February-13
  • Gender:Male
  • Location:Natick, MA
  • Interests:Travel
    Cooking
    Brewing
    Hiking

Posted 2020-May-20, 16:39

View Posthelene_t, on 2020-May-20, 16:17, said:

The 1st should be Deal, not Dealer, I think?


Second actually, but thanks very much for the correction
Alderaan delenda est
0

#7 User is offline   MaxHayden 

  • PipPip
  • Group: Members
  • Posts: 34
  • Joined: 2019-August-25

Posted 2020-May-20, 23:31

View Posthrothgar, on 2020-May-20, 16:13, said:

I think that the project that you are suggesting is incredibly ambitious.

If I were going to kick off this sort of approach, I suspect that I would focus on trying to train a Deep Learning program to play bridge and then give it a couple weeks to really learn how to play bridge rather than approach this analytically.

If you do want to solve this for yourself, I'd recommend focusing on a very narrowly constrained problem and see whether you're able to come up with a satisfactory solution. I personally would suggest trying to compare the efficiency of various auction termination mechanisms following shape resolution in game forcing relay auctions.

If you can come up with really convincing results to this (relatively) simple problem you will have accomplished something significant (and given you a strong foundation to consider other more complicated questions)

And, if you can't solve this (relatively) simple problem then you might need to rethink your approach


First, thanks for your help thus far.

I'm curious about the two approaches you suggested. I'm unclear on what the deep learning gets me. Do you mean that it includes bidding? And what kind of analysis are you envisioning? I could see trying to deep learn the conditional probability itself or to solve the particular problem you are talking about (assuming that some more rudimentary automated method doesn't work). But what does something that actually plays bridge add for that complexity?

As for the relay/bidding termination; my assumption was that this in itself was overly ambitious and that I should first try to quantify some more basic questions and systematically check my core assumptions. (Maybe you intended this to be included in the problem as you stated it?)

For example, right now our best hand evaluation methods have a standard error of .9 tricks. And a more in-depth method that includes known adjustments that you can't practically do live would be better by some unknown amount. You can do something similar for the offense vs defense aspect. Same with some of these other things. So that's the baseline.

For my whole idea to even be meaningful, I need some example scenario where there's a meaningful effect size. If I can't get a significantly lower variance by holding part of the deal constant and then rescrambling the rest, the whole enterprise is pointless. My assumption is that because a simple finesse is .5 tricks that this is a lower limit on what can be accomplished in most cases. If that's right there's a lot of potential here. But if it turns out that an ambitious unconditional model results in .7 tricks of error and that I can't find a scenario where the error inherent in double-dummy results upon rescrambling is less than .65, then this task won't even be fruitful. The impact needs to be something that produces a reasonable amount of MPs or IMPs over a typical tournament.

So I need to first identify some low-hanging fruit before I even break out the more complex stuff. Otherwise I can't be sure that I'm even in a position to make a convincing case. (Plus having some kind of coherent documentation of our baseline knowledge and the areas most in need of improvement seems valuable in its own right since it doesn't seem like such a thing exists right now.)

Is this a good line of thought or do you think there's a way to short-circuit this and jump into the problem itself?
0

#8 User is offline   njgeorge 

  • Pip
  • Group: Members
  • Posts: 1
  • Joined: 2020-May-21

Posted 2020-May-21, 09:52

Not sure if you are familiar with The Mathematical Theory of Bridge originally by Emile Borel and Andre Cheron but translated into English by Alec Traub

Originally written in 1939 but covers the subject pretty comprehensively. No help with software (written in 1939) but begins with the basics, and brings you along. 188 pages of discussion, with about 270 pages of notes, tables and probabilities behind the discussion.
0

#9 User is offline   miss plays 

  • Pip
  • Group: Members
  • Posts: 2
  • Joined: 2020-May-21

Posted 2020-May-21, 18:15

I have been developing an iOS app that does everything you are looking for, as well as helping with conventions and what various bids mean in different systems. If you are interested in taking a look before I (ever) get to publishing it, and are a little technically proficient (you would need to pull my code from a public GitHub server and run it using Xcode, a free Apple product), then I can send you a link. Message me directly.
0

#10 User is offline   hrothgar 

  • PipPipPipPipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 15,372
  • Joined: 2003-February-13
  • Gender:Male
  • Location:Natick, MA
  • Interests:Travel
    Cooking
    Brewing
    Hiking

Posted 2020-May-22, 02:50

View Postmiss plays, on 2020-May-21, 18:15, said:

I have been developing an iOS app that does everything you are looking for, as well as helping with conventions and what various bids mean in different systems. If you are interested in taking a look before I (ever) get to publishing it, and are a little technically proficient (you would need to pull my code from a public GitHub server and run it using Xcode, a free Apple product), then I can send you a link. Message me directly.


Why would you ever want to do large scale statistical modeling and monte carlo simulation on IOS (or a device that runs on a battery)

I suspect that there is some confusion about the nature of the project
Alderaan delenda est
0

#11 User is offline   pigpenz 

  • PipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 2,553
  • Joined: 2005-April-25

Posted 2020-May-22, 10:36

View Postnjgeorge, on 2020-May-21, 09:52, said:

Not sure if you are familiar with The Mathematical Theory of Bridge originally by Emile Borel and Andre Cheron but translated into English by Alec Traub

Originally written in 1939 but covers the subject pretty comprehensively. No help with software (written in 1939) but begins with the basics, and brings you along. 188 pages of discussion, with about 270 pages of notes, tables and probabilities behind the discussion.

this book is a beast good luck with it
0

#12 User is offline   miss plays 

  • Pip
  • Group: Members
  • Posts: 2
  • Joined: 2020-May-21

Posted 2020-May-22, 18:59

View Posthrothgar, on 2020-May-22, 02:50, said:

Why would you ever want to do large scale statistical modeling and monte carlo simulation on IOS (or a device that runs on a battery)

I suspect that there is some confusion about the nature of the project


Yes, generating thousands of heavily constrained hands can take a minute or two, but it's 2020, the iPad can handle it.

You can also run it on MacOS.

If you'd like to see my code, just let me know, I can link you up.

There's no confusion, just hundreds of thousands of lines of code. It's a pet project of mine, just for fun.
0

#13 User is offline   hrothgar 

  • PipPipPipPipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 15,372
  • Joined: 2003-February-13
  • Gender:Male
  • Location:Natick, MA
  • Interests:Travel
    Cooking
    Brewing
    Hiking

Posted 2020-May-23, 01:38

View Postmiss plays, on 2020-May-22, 18:59, said:

Yes, generating thousands of heavily constrained hands can take a minute or two, but it's 2020, the iPad can handle it.

You can also run it on MacOS.

If you'd like to see my code, just let me know, I can link you up.

There's no confusion, just hundreds of thousands of lines of code. It's a pet project of mine, just for fun.


There's nothing wrong with having a pet project, however, if I am trying to do serious work then at a bare minimum I am going to be running a serious workstation with a big ass processor, a bunch of RAM, and a couple GPUs.
And even then, I am often going to be annoyed at the amount of time a bunch of this work is taking and wishing that I could use the clusters from work for vanity projects.

A mobile device is the wrong tool for the job. (Plus, once again, BATTERY LIFE)

And this is before all sorts of issues with validating third party code, PRNGs, and the like.
Alderaan delenda est
0

#14 User is offline   helene_t 

  • The Abbess
  • PipPipPipPipPipPipPipPipPipPipPip
  • Group: Advanced Members
  • Posts: 17,068
  • Joined: 2004-April-22
  • Gender:Female
  • Location:UK

Posted 2020-May-23, 06:54

Hi Miss Plays, welcome to the forum!

Sounds like an interesting project, I am curious to hear about the functionality of your app.

Don't worry about hrothgar's criticism, he is the official BBF contrarian so it's his job to trash your ideas. Don't take it personally, and above all, keep posting :)
The world would be such a happy place, if only everyone played Acol :) --- TramTicket
0

Page 1 of 1
  • You cannot start a new topic
  • You cannot reply to this topic

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users