Whatever Josh, the simulation results were very different after changing a queen to a king. I pointed out that there was a big difference, this does not mean that I was making a point. I did not claim that the first simulation said we should pass nor did I claim that the second simulation says that we should invite with that hand.
In fact, I pointed out some factors why these simulations should be regarded somewhat skeptically. Fluffy pointed out one factor of why the results of the second simulation might be so different from the first simulation, I agree that that was a very good observation.
Also, I didn't run the second simulation to make a point, I ran it because (a) Roger asked me to run the same simulation for that exact hand, and (

I thought that perhaps the first simulation results were more negative than they should be. Running the same search for a hand where my intuition says we should clearly invite can be a helpful: if the simulation results again suggested that one should pass then that would be reason to be even more skeptical about the first simulation. It turned out that this was not the case but that doesn't mean that either simulation should be accepted as enough evidence for the best call at the table, and I certainly didn't say they should.
It remains an interesting question how closely simulation results match real world results. I've read some work that was done in this direction but I never found what I was looking for. There is data suggesting that for large sets of hands (i.e. different hands dealt to us), the average number of tricks a double dummy simulation gives is quite close to the average number of tricks taken in real life, and that in real life, declarer tends to take more tricks in partscore contracts, fewer tricks in slams and about the same number of tricks in games, and for each of these situations the margin is not very large.
But that does not mean that for any specific hand the double dummy results will be close to the at-the-table results, it is easily possible that for some hands the real life results tend to be much more in favor of the defense while on others the results tend to be much more in favor of declarer. So I remain very skeptical about double dummy simulations.
I think perhaps the best use of simulations is as feedback for your intuition. I first try to guess how likely game or slam is and then I run a double dummy simulation to see if it agrees with my intuition. If not, I will look at few dozen hands manually to see if I can learn something. Sometimes the hands dealt to partner are very unrealistic and I will have to adjust the specifications. More often I notice some factors I didn't consider before, and hopefully I will have learned something.
Please note: I am interested in boring, bog standard, 2/1.
- hrothgar