[dake50]

Feels this starts at the wrong place. Start with "what really matters?"
Eg does 2hcp in 5332 matter? How much? At the 0.0003% level? if yes, why wasn't that precision postponed until the 0.5% effects are found and schemed?
"What's in the cards" before some edifice of bidding.!

[Free]

I think the definition of "(c1<clubs<c2) & ...... & (p1<hcps<p2)" is easy to program, but in reality it's useless for system design.

Just look at a simple example, the minimulti:
0-4♣, 0-4♦, 0-6♥, 0-6♠, 6-9HCP

Suppose you also play Muiderberg (2M = 5M, 4+m):
0-5♣, 0-5♦, 0-5♥, 0-5♠, 6-9HCP

The program will not be able to decide which opening bid it should choose when holding a 5-3-3-2 with 7HCP, while in fact it should just pass!

It's important to have clear definitions available, so you don't have multiple options.

If you can put it as one of multiple options, you'd have a lot better description:
minimulti:
- 0-4♣, 0-4♦, 0-3♥, 6♠, 6-9HCP
- 0-4♣, 0-4♦, 6♥, 0-3♠, 6-9HCP
Muiderberg 2♠:
- 4-5♣, 0-3♦, 0-3♥, 5♠, 6-9HCP
- 0-3♣, 4-5♦, 0-3♥, 5♠, 6-9HCP
Now we don't have a problem anymore, and the 5-3-3-2 with 7HCP won't choose any of these openings.

Another very simple example is balanced hands. You could say 2-5 cards in all suits, but then you'll allow 5422's (even with 5-4M) which is probably not what you want. A simple way to describe all balanced hands is to order your distribution high to low (the general form) and you need at least "3332". With your definition you need 4 options:
- 2-4♣, 3-5♦, 3-5♥, 3-5♠, 15-17HCP
- 3-5♣, 2-4♦, 3-5♥, 3-5♠, 15-17HCP
- 3-5♣, 3-5♦, 2-4♥, 3-5♠, 15-17HCP
- 3-5♣, 3-5♦, 3-5♥, 2-4♠, 15-17HCP

The simple definition is good enough to start with, as long as you allow multiple meanings.

[helene_t]

bab9, on Jun 1 2010, 03:43 AM, said:

Are you going to allow the system to play in a Moysian fit?

The system could certainly "decide" to pass what is known to be a Moysian, or what might be a Moysian. But I think I would say that the utility of the information that I have at least three spades is zero if partner is known to have exactly four spades. But if partner is known to have 4+ spades, the expected utility of the information that I have 3+ spades is positive since he might have five. Anyway, this is a parameter in the algorithm that should be optimized by the genetic algorithm (for example).

free said:

I think the definition of "(c1<clubs<c2) & ...... & (p1<hcps<p2)" is easy to program, but in reality it's useless for system design.

Yeah, this would probably turn out to generate rather ineffective systems.

I can live without the multi, but for example the 1♣ opening in SA would have to map to 11 terminal nodes in the tree:
- 11-21 hcps, 6+ clubs, 5- d/h/s
- 11-21 hcps, 5 clubs, 4- h/s, 1- d
- 11-21 hcps, 5 clubs, 4- h, 1- s, 3 d
- 11-21 hcps, 5 clubs, 4- s, 1- h, 3 d
- 17-21 hcps, 5 clubs, 4- h/s, 4 d
- 12-14 hcps, 4 clubs, 4- h/s, 3- d
- 12-14 hcps, 3 clubs, 4- h/s, 3- d
- 12-14 hcps, 5 clubs, 3- d/h/s
- 18-19 hcps, 4 clubs, 4- h/s, 3- d
- 18-19 hcps, 3 clubs, 4- h/s, 3- d
- 18-19 hcps, 5 clubs, 3- d/h/s

The decision tree with one-to-one mapping between terminal nodes and "possible" calls is just a first step, to make it useful I would have to either use a completely different formalism, or to allow calls to map to multiple nodes.

Maybe there is a simple formalism that would cover a more useful range systems.

As for the idea of allowing general forms such as "3332" as opposed to "[3332]", I think it is too narrowly targeted at balanced hands. I mean would you want an opening dedicated to, say, 5431? I know some systems have a concept of a generic 4441 but I think that is a suboptimal choice that has been made because it is easy to remember. I think it is arbitrary to bundle [4441] with [4144], you might as well bundle [4441] with, say, [5521].

[zenko]

helene_t, on May 31 2010, 05:25 AM, said:

There will be a one-to-one relation between legal "possible" calls and terminal nodes. By "possible" I mean that for example pass when partner is unlimited, or 7NT when our combined assets are limited to 23 HCPs, are not included.

AKxxxxxxxx
x
x
x


x
Axxx
Axxx
Axxx

laydown 7NT, with 4 HCPs to spare!

[bab9]

helene_t, on Jun 1 2010, 03:51 AM, said:

bab9, on Jun 1 2010, 03:43 AM, said:

Are you going to allow the system to play in a Moysian fit?

The system could certainly "decide" to pass what is known to be a Moysian, or what might be a Moysian. But I think I would say that the utility of the information that I have at least three spades is zero if partner is known to have exactly four spades. But if partner is known to have 4+ spades, the expected utility of the information that I have 3+ spades is positive since he might have five. Anyway, this is a parameter in the algorithm that should be optimized by the genetic algorithm (for example).

free said:

I think the definition of "(c1<clubs<c2) & ...... & (p1<hcps<p2)" is easy to program, but in reality it's useless for system design.

Yeah, this would probably turn out to generate rather ineffective systems.

I can live without the multi, but for example the 1♣ opening in SA would have to map to 11 terminal nodes in the tree:
- 11-21 hcps, 6+ clubs, 5- d/h/s
- 11-21 hcps, 5 clubs, 4- h/s, 1- d

If you are going to use a genetic algorithm, what form will your cost function take? Will it be based on double dummy, or actual results? How big is your sample size likely to be?

If you are going to use a genetic algorithm, why not let it determine the ranges and card counts, as an alternative to starting with a decision tree?

[helene_t]

bab9, on Jun 2 2010, 03:24 AM, said:

If you are going to use a genetic algorithm, why not let it determine the ranges and card counts, as an alternative to starting with a decision tree?

The problem is that there are millions of decision trees (one for each possible situation that could arise during the auction), so you can't optimize individual ranges.

You need some general principles that guide the construction of the decision trees. Those general principles can have parameters which can be optimized. For example, the utility of knowing that p has 3+ in a major when you have four cards yourself could be a parameter.

Quote

what form will your cost function take?

I was thinking of using a very crude heuristic, say tricks=trumps + (hcps-20)*0.4, where trumps is 6.5 for NT contracts. Then IMPing the achieved result to "PAR", or maybe just counting total points. Obviously something could be thought of that would be a lot better but I think this is a minor issue compared to all the other problems this problem will/would face.

Quote

How big is your sample size likely to be?

I dunno, what would you suggest? Say I have computer resources for 100,000 auctions, would something like 3333 generations of 30 individuals be reasonable?

[hotShot]

I would like to point out, that the suggested approach will lead to a "computer optimized" system and not to a "computer generated" system.

The parameterization is in fact the core of a system, what would be left to the computer would be to optimize some ranges.
The same is to say about fitness functions.

Of cause computers can't generate systems yet and I doubt that they will be able to do anything like that in the near future, so a computer optimized system is the only realistic target reach.

I would suggest to start with a reduced decision tree / sample set e.g.
1M openings and follow up bidding with fit.

[hotShot]

If we could train a neural network to bid, we would still end up with the problem to extract the "rules" it uses to determine the bids, especially to get them in a human usable form.

[NickRW]

bab9, on May 31 2010, 02:57 AM, said:

- double dummy results
- actual results by experts
- something else?

Strictly speaking, single dummy. Double dummy results tend to over estimate slam potential on strong hands and under estimate part score hands (in the former case, the declaring side has most of the top cards, is in control and will always drop/finesse or squeeze correctly - in the latter case the defence has enough cards to do some damage and will always get off to the right lead etc).

[hrothgar]

Hi Helene

Haven't given all that much thought to this, however, this sounds a lot more like a genetic algorithm rather than a decision tree...

From my perspective, the right starting point is defining how you plan to measure fitness, after which you can starting considering initial populations, followed by mutations and recombinations.

[helene_t]

Hi Richard,

I think those issues you mention are the relatively easy ones.

What I find difficult is the "physiology" of the bidding system: how to map it's "genes" (mutable characteristics) to decision making in concrete situations.

When I design conventions and systems "by hand", I do it quite differently from what I have sketched above. The manual method is simpler in that I will assume certain symmetries (for example, the follow-ups after a natural 1♥ opening are similar to those after a 1♠ opening). But it is more complex in that I will not sequentially optimize the meaning of the first step, then the next step etc., nor will I separate the informational part (which information to convey) from the semantic part (how to encode that information).

I wonder if it would be better to let the computer-based system should mimic the manual procedure more closely.

[hotShot]

Without thinking this to the end I suggest something like that:

The way I understand this, is that the "DNA" needs to be the full decision tree aka the whole bidding sequence.
So you automatically generate a starting generation with decision trees that contains (random) combinations from limit raises, preemptive raises, strong raises, Bergen, Jacoby, Mini-Splinter, Splinter, ... ,passing if a termination rule is true.
To each decision tree generate several individuals with different HCP / suit length intervals for each decision.

As fitness function take the percentage of the testset boards that fit the function decision tree difference_in_total_points (decision tree result , par result) < (something like 199).

To build the new generation take the average/combined ranges of identical decisions in the tree. (The problem can/will occur that the decision tree will not have ?disjunct? decisions!)
Or perhaps transfer complete nodes.

[bab9]

NickRW, on Jun 2 2010, 11:29 AM, said:

bab9, on May 31 2010, 02:57 AM, said:

- double dummy results
    - actual results by experts
    - something else?

Strictly speaking, single dummy. Double dummy results tend to over estimate slam potential on strong hands and under estimate part score hands (in the former case, the declaring side has most of the top cards, is in control and will always drop/finesse or squeeze correctly - in the latter case the defence has enough cards to do some damage and will always get off to the right lead etc).

Is there a database containing hands and single dummy results available?

[bab9]

helene_t, on Jun 2 2010, 03:44 AM, said:

bab9, on Jun 2 2010, 03:24 AM, said:

If you are going to use a genetic algorithm, why not let it determine the ranges and card counts, as an alternative to starting with a decision tree?

The problem is that there are millions of decision trees (one for each possible situation that could arise during the auction), so you can't optimize individual ranges.

You need some general principles that guide the construction of the decision trees. Those general principles can have parameters which can be optimized. For example, the utility of knowing that p has 3+ in a major when you have four cards yourself could be a parameter.

What if the problem was separated into a number of developmental stages. For example, instead of looking at HCP and distribution, why not initially look at just the distribution (ignoring HCP) to ensure that the algorithm can find the optimal suit fit, or determine if no trumps is the better option.

This may result in a set of possible solutions that could be used as a basis for the genetic algorithm when both HCP and distribution are taken into account.

[1eyedjack]

I don't know squat about programming, but as a human I have been down the evolutionary approach in system design, wherein marginal changes are trialled in order to ascertain whether the result is superior. This approach suffers from one major drawback: You might conclude that you have optimised the system at the point when you are unable to make a small change to the system such as to provide an overall benefit. The problem with that is that the changes may not be large enough to identify a benefit that could be available by making a much larger or more fundamental change.

Visualise your achievement in system design as graphically presented on the face of a mountain. The higher up the mountain you climb, the better the system. Being blind, you feel about with your foot and notice that (say) travelling North leads uphill, so you move in that direction until travelling in any direction leads downhill. You conclude that you have reached the summit, and indeed so you have. But then if you take off the blindfold, in the distance you see a higher mountain altogether. If your baby steps had been a couple of miles apart you might have detected it. Translating this effect into a self-programming computer sounds to me like a monumental task.

[helene_t]

Yeah, your sample size is too small. And if you try the new system out for years to obtain a good sample size, you will get confounding due to changing fields, changing partners, or improvement of your own game irrespective of system.

My feeling is that to make sense of real life experience with bidding methods you need to assess not only how often it gave good results / bad results, but also why it gave those good results. Most good results may turn out not to be evidence for the strength of the system but could have been obtained with the reference system as well. Or maybe a flaw in the system took you to an inferior spot which happened to give a good result due to an unlikely split. But I may be wrong about this since it is very hard to make all these assessment objectively. Probably the only way to avoid an emotional bias is simply to compare the MPs or IMPs you get with one system to what you get with the other system.

However, experience can be combined with theoretical assessment. Say you change to a strong club system and from a theoretical POV there could be an issue with opps preempting over your strong club. You can identify some situations that are likely to take you to a bad spot, find out how often they come up either by back-of-the-envelope probability calculations or by simulations, and then inspect simulation results manually to get an idea of how often you would actually end up in a wrong spot.

All this knowledge can be augmented by your experience when playing the system at the club: maybe we have a great defense against enemy preempts in theory, but in practice it often leads to bad results because we have difficulties dealing with opps with unknown preempt style. Maybe lots of boards that should have given us a bad result give us a good result because opps preempt too much, too little, or are not on the same wavelength.

And then there are issues that can hardly be dealt with theoretically. Such as: Do we feel comfortable playing this new style? Maybe it turns out that having to memorize complex conventions and explain them to annoyed opps take too much focus away from the really important things like judgement.

[NickRW]

bab9, on Jun 9 2010, 02:23 AM, said:

NickRW, on Jun 2 2010, 11:29 AM, said:

bab9, on May 31 2010, 02:57 AM, said:

- double dummy results
    - actual results by experts
    - something else?

Strictly speaking, single dummy. Double dummy results tend to over estimate slam potential on strong hands and under estimate part score hands (in the former case, the declaring side has most of the top cards, is in control and will always drop/finesse or squeeze correctly - in the latter case the defence has enough cards to do some damage and will always get off to the right lead etc).

Is there a database containing hands and single dummy results available?

There are couple of articles
here (linked about 2/3rds of the way down the page) about the perils of both double and single dummy data.

Suggest (could be wrong) that to get a worthwhile database of single dummy data you need to generate a large quantity (like 6 figure number of hands at least) of results of relatively decent bots playing against one another.

Dunno exactly how you do that - the organisers of the computer world championships specify a standard interface so that bots can communicate over a network - and the makers of said bots ought, in theory, to be interested in helping to produce such a database (if they don't have one already!) provided they were given a copy of it - I would have thought...

Nick

[helene_t]

Just making a little experiment with a neural network instead, to get something running quickly.

After 300 generations it still bids horribly ineffecient. Decisions to leave in partner's double are generally reasonable but the constructive bidding is ridiculous and they have a bad habit of redoubling everything and partner doesn't rescue.

They almost never open at a higher level than 1♠, and you end up with bizarre systems like
pass=basically 0-7 or 18+
1♠=8-14 points with 4-6 clubs and either 1- hearts or exactly 4 hearts.
Maybe something like that could be effective if the follow-ups worked. Would be interesting to see.

Now I have a go with 100,000 generations, I wonder if something sensible will materialize. I am hoping for something really bizarre with many multi-bids, forcing passes etc

[gnasher]

helene_t, on Jun 19 2010, 02:36 PM, said:

pass=basically 0-7 or 18+

That's also featured in human-designed systems.

Quote

1♠=8-14 points with 4-6 clubs and either 1- hearts or exactly 4 hearts.

even that is vaguely similar to the TRS major-suit openings

[hrothgar]

gnasher, on Jun 19 2010, 05:26 PM, said:

helene_t, on Jun 19 2010, 02:36 PM, said:

pass=basically 0-7 or 18+

That's also featured in human-designed systems.

Quote

1♠=8-14 points with 4-6 clubs and either 1- hearts or exactly 4 hearts.

even that is vaguely similar to the TRS major-suit openings

I would be highly amusing if the Neural Network produced something akin to the Vulcan Variable Pass