[hrothgar]

Hi All

During yesterday's VuGraph, Fred mentioned that BBO's ISP was encountering some problems, leading to connectivity problems. I had the chance to think about this over night and reached a rather interesting conclusion: BBO's currently implementation has to potential to create positive feedback loops within traffic patterns. So long as the total amount of BBO traffic represents a very small proportion of the total traffic for an ISP this is unlikely to create any serious problems. However, the aggregate traffic on BBO grows to become a sizable component of an ISP'd traffic stream, you have the potential for some SERIOUS problems.

All this is based on one very simple observation:

The bandwidth utilization of the BBO application is time sensitive. When a connection is first initialized, the BBO server transfers relatively large amounts of data to the BBO client descriing the "state" of the lobby, etc. As I understand matters, this transfer is responsible for the delays that some users experience in trying to connect across low bandwidth links.

Normally, this doesn't create a serious problem, but consider the impact if BBO if operating on a network link that is experiencing congestion problems:

1. Network congestion causes a group of players to drop their connections.
2. These players NATURALLY immediately try to reconnect, causing a large amount of data to get dumped from the server down to the clients.
3. This immediately worsens the congestion problem, causing more conections to get dropped.

This is a classic positive feedback loop at work.

I don't have the data necessary to know whether this is happening or not, however, it is definitely worth considering.

This post has been edited by hrothgar: 2004-October-26, 14:22

[uday]

Yes, it sometimes goes the way you describe. There are other moving parts (lots of them) as well.

The problems w/sluggishness we've been spotting lately are related, but not identical. We're consuming too much bandwidth in our area of the hosting providers network (choking it off at peak), and the scheduled move ( tonight tonight @ 12:30 AM EST) rates to make things better since our new location has greater capacity.

[hrothgar]

uday, on Oct 26 2004, 11:15 PM, said:

Yes, it sometimes goes the way you describe. There are other moving parts (lots of them) as well.

The problems w/sluggishness we've been spotting lately are related, but not identical. We're consuming too much bandwidth in our area of the hosting providers network (choking it off at peak), and the scheduled move ( tonight tonight @ 12:30 AM EST) rates to make things better since our new location has greater capacity.

Hi Uday,

Extra bandwidth is always a good thing. In this case, a fatter pipe will make it much less likely that local bandwidth spikes will cause an "explosive" condition. With this said and done, the underlying dynamic is still present and could raise its ugly little head under a variety of conditions. For example, a synchronized event in which a large number of users connected at the same time (start of a VuGraph, for example) could also trigger the same type of occurence.

If/When you are reworking the client, you might want to consider mechanisms to spread out the bandwidth, even if it does cause mild increases in latency...

[hotShot]

I assume if 1000 kibs log in shortly before a vugraph sesssion starts this could let to a bandwidth shortage..

I wonder how much bandwidth you need.
I think one player/kib needs about 1.5 MB/h downstream.
I see about 5100 Users online resulting in a 7.5 GB/h bandwidth top performance.
More than a 16Mbit connection can take.
Lets assume an average of 2000 user/h 24 h a day
that results in more than 2TB (Terabyte) per month.

Whow!

[hrothgar]

hotShot, on Oct 27 2004, 07:59 PM, said:

I assume if 1000 kibs log in shortly before a vugraph sesssion starts this could let to a bandwidth shortage..

I wonder how much bandwidth you need.
I think one player/kib needs about 1.5 MB/h downstream.
I see about 5100 Users online resulting in a 7.5 GB/h bandwidth top performance.
More than a 16Mbit connection can take.
Lets assume an average of 2000 user/h 24 h a day
that results in more than 2TB (Terabyte) per month.

Whow!

Important rule of thumb:

"Average" traffic loads are pretty much meaningless when it comes to network design. Its VERY easy to design system that feature deterministic traffic patterns.

Burst capacity and feedback is where life gets interesting...

[hotShot]

hrothgar, on Oct 27 2004, 06:19 PM, said:

....
"Average" traffic loads are pretty much meaningless when it comes to network design. Its VERY easy to design system that feature deterministic traffic patterns.

Burst capacity and feedback is where life gets interesting...

Well i don't have the data for a full analysis,
but a monthly transfer volume of more than 2TB is interesting.