Limiting upload bandwidth usage

Thread view:

Enable EMail Alerts

Start New Thread

Thread rating:

Patrick Machielse - 23 Aug 2007 15:35 GMT

I'm currently researching a program that will upload a lot of files in
the background. By a lot I mean: 10+ GB, so the upload process may take
considerable time (anno 2007).

Since this program should truly be a 'background' program, and its
operation should not be noticible to the user, it should ideally not
hyjack the all the upload bandwidth. While discussing the pros, the
cons, and the hows I had the following ideas:

- Let's not whorry about it and let OS X figure it out. The system is in
charge of distributing the available bandwidth, and smart people have
already spent a lot of time solving this problem.

However, in practice programs that use as much bandwidth as they can
aren't un-noticible... Having a program 'hogging' all bandwith can
become quite annoying. Also, file sharing programs like Transmission let
you set bandwidth limits, which seems to help. Therefore, limiting the
bandwidth seems to be a good thing.

- If we limit bandwidth usage, should we set it at a fixed amount, or
can we set it to a percentage of the available bandwidth? A fixed amount
would require user intervention, which we want to avoid. Also it would
require knowledge that the user probably doesn't have. Ideally the
solution should be 'automatic'.

I noticed a post on CocoaBuilder where someone simply waited in an
NSURLConnection delegate method to limit the bandwidth usage, which
seemed to work. Are there lower level instruments?

- Alternatively, would it be possible to set some kind of 'service of
quality' parameter on the network usage generated by my application, so
that my app only uses the network when there are no other 'takers'. I
think this would be the ideal situation.

If anyone can/wants to share some insights, I'd appreciate it.

patrick

Reply to this Message

Michael Ash - 23 Aug 2007 16:51 GMT

> - Let's not whorry about it and let OS X figure it out. The system is in
> charge of distributing the available bandwidth, and smart people have
[quoted text clipped - 5 lines]
> you set bandwidth limits, which seems to help. Therefore, limiting the
> bandwidth seems to be a good thing.

This fails because of an interaction between two design features of the
internet.

TCP has bulit-in rate limiting which works by backing off transmission
rates when congestion is encountered. It detects congestion by watching
for lost packets. When packets are lost it assumes it's because the
network is congested. Conversely, when no packets are lost it assumes that
it can go faster.

Most internet routers try pretty hard not to drop any packets and as such
have significant packet queues for use when congestion occurs. These are
often a couple of seconds' worth of transmission. When they are full,
considerable latency is added to your connection which makes it seem
extremely slow.

The combination of these two strategies means that TCP will fill up all of
your queues before deciding that it's going fast enough, so for the
duration of a full-blast upload, you'll experience latency on the order of
seconds on your connection, which makes it pretty unpleasant to use.

OS X in particular and the internet in general do a fine job of
distributing bandwidth between things that need it. What they don't do so
well at is avoiding huge latency spikes when full capacity is reached. As
Stuart Cheshire says, "It's the latency, stupid."

> - If we limit bandwidth usage, should we set it at a fixed amount, or
> can we set it to a percentage of the available bandwidth? A fixed amount
> would require user intervention, which we want to avoid. Also it would
> require knowledge that the user probably doesn't have. Ideally the
> solution should be 'automatic'.

I believe the best strategy would be to measure the latency to the remote
site periodically, and back off your transmission speed when it starts to
spike. This will end up being pretty complicated to implement because
you'll need to consider a lot of different scenarios, like what happens if
the link between you and the server suddenly gets twice as much latency on
it for reasons unrelated to you. Or what happens if something else fills
up the queues, making it look like your download has no effect, and
thereby filling up the pipes even worse. But if you can manage to keep the
connection latency within a reasonable factor of the "idle" latency then
you'll avoid making the connection unusable.

No matter what you do I would recommend having a manual override somewhere
in case your strategy isn't appropriate for somebody.

> I noticed a post on CocoaBuilder where someone simply waited in an
> NSURLConnection delegate method to limit the bandwidth usage, which
> seemed to work. Are there lower level instruments?

Obviously it will depend on what you're using to upload.

I don't see any way to limit *upload* bandwidth usage in an
NSURLConnection method. In fact I don't even see a way to measure it.

If you can require 10.4 (and you must, because 10.3 has no way to upload
10GB of data over NSURLConnection anyway) then you can use the
setHTTPBodyStream: method of NSMutableURLRequest and then write a custom
NSInputStream with bandwidth measurement and rate limiters built in. Rate
limiting in this case can be accomplished by blocking the read method for
some time before returning.

> - Alternatively, would it be possible to set some kind of 'service of
> quality' parameter on the network usage generated by my application, so
> that my app only uses the network when there are no other 'takers'. I
> think this would be the ideal situation.

This is not possible in general. The problem is that your Mac is not
necessarily on the connection bottleneck, and internet devices in general
are not smart enough to be able to do this sort of thing.

For example, my desktop Mac is connected over a 100Mbit ethernet
connection to an Airport Express. It in turn connects wirelessly to
another Airport Express, and that wireless link tops out in real-world
terms at around 10Mbit. Then it hits my cable modem, with a blazing
0.4Mbit upload.

Obviously OS X can do whatever it wants to try to equitably share that
100Mbit connection that it can actually see, but nothing it does is going
to stop an app on my computer from stomping my wife's web browsing
experience.

In theory the cable modem should be able to prioritize traffic based on
some sort of tag, and so shove the low-priority upload stuff to the back
of the queue. In practice the devices simply aren't intellilgent enough
for that, and will treat all traffic equally.

(Actually this last bit isn't true. My phone runs through that cable modem
too, and it's smart enough to prioritize it well above anything else it
transmits. But this doesn't generalize, so you can't have it deprioritize
some traffic more than others.)

Signature

Michael Ash
Rogue Amoeba Software

Reply to this Message

Patrick Machielse - 24 Aug 2007 16:37 GMT

> The combination of these two strategies means that TCP will fill up all of
> your queues before deciding that it's going fast enough, so for the
> duration of a full-blast upload, you'll experience latency on the order of
> seconds on your connection, which makes it pretty unpleasant to use.

Aha!

> > - If we limit bandwidth usage, should we set it at a fixed amount, or
> > can we set it to a percentage of the available bandwidth?
[quoted text clipped - 7 lines]
> No matter what you do I would recommend having a manual override somewhere
> in case your strategy isn't appropriate for somebody.

This sounds like a reasonable approach. I'll make a mental note of the
word 'latency'...

> If you can require 10.4 (and you must, because 10.3 has no way to upload
> 10GB of data over NSURLConnection anyway) then you can use the
> setHTTPBodyStream: method of NSMutableURLRequest and then write a custom
> NSInputStream with bandwidth measurement and rate limiters built in. Rate
> limiting in this case can be accomplished by blocking the read method for
> some time before returning.

OK, that sounds like a useful approach, I'm definately going to look
into it. The upload will not be 1 file, so there is room to do some
'meta' accounting on the individual transmissions and maybe draw
conclusions from that. Regardless, it seems that we'll always be at the
mercy of how well behaved all other network using programs are...

Anyway, thanks for the elaborate overview of the causes and solutions to
bandwidth congestion! It's certainly enhanced my understanding of the
problem :-)

patrick

Reply to this Message

Michael Ash - 24 Aug 2007 18:09 GMT

>> I believe the best strategy would be to measure the latency to the remote
>> site periodically, and back off your transmission speed when it starts to
[quoted text clipped - 7 lines]
> This sounds like a reasonable approach. I'll make a mental note of the
> word 'latency'...

A great discussion of bandwidth versus latency and how they each affect
a user's internet experience can be found here:

http://www.stuartcheshire.org/rants/Latency.html

It is terribly dated in many ways, talking about ISDN and Geoport telecom
adapters, and discussing cable modems entirely in the hypothetical sense,
but the overall ideas are all unchanged.

There's lots of other great information on his site which is great for
anyone who wants to understand networking at a deeper level.

>> If you can require 10.4 (and you must, because 10.3 has no way to upload
>> 10GB of data over NSURLConnection anyway) then you can use the
[quoted text clipped - 8 lines]
> conclusions from that. Regardless, it seems that we'll always be at the
> mercy of how well behaved all other network using programs are...

Of course there's nothing preventing you from doing accounting during the
transmissions either.

A naive approach might be to have some kind of "ping" you can do to the
server, which might just be a quick HTTP request, or a UDP packet sent to
some kind of echo service. Do a couple of those before you start to
establish a baseline, then decide on a threshold. Start your transmission,
and keep pinging once every X seconds. If you're under the threshold,
increase your rate by some proportion. If you're over, decrease it. In the
space between transmissions you can do more pinging to make sure your
baseline is still valid, or you can even pause a long transmission to do
it. This is fairly complicated but the real-time feedback will help you
deal with changing network conditions, like what would happen if the user
starts a big upload of his own while yours is going on.

> Anyway, thanks for the elaborate overview of the causes and solutions to
> bandwidth congestion! It's certainly enhanced my understanding of the
> problem :-)

You're welcome. I hope it helps, even if you end up deciding that the
complicated approach is perhaps too much.

Signature

Michael Ash
Rogue Amoeba Software

Reply to this Message

Tom Harrington - 23 Aug 2007 17:38 GMT

> - If we limit bandwidth usage, should we set it at a fixed amount, or
> can we set it to a percentage of the available bandwidth? A fixed amount
[quoted text clipped - 5 lines]
> NSURLConnection delegate method to limit the bandwidth usage, which
> seemed to work. Are there lower level instruments?

Take a look at libcurl, which is included on Mac OS X. It includes
options for setting limits on upload and download speeds. I believe it
basically works similar to what you describe-- keep track of how many
bytes have been transferred and how long it has taken, and pause as
needed to approximate the limit as closely as possible.

Signature

Tom "Tom" Harrington
MondoMouse makes your mouse mightier
See http://www.atomicbird.com/mondomouse/

Reply to this Message

Patrick Machielse - 24 Aug 2007 16:37 GMT

> options for setting limits on upload and download speeds. I believe it
> basically works similar to what you describe-- keep track of how many
> bytes have been transferred and how long it has taken, and pause as
> needed to approximate the limit as closely as possible.

Thanks for the pointer, I'll look into it.

patrick

Reply to this Message

Mac Forum / Programming / Mac Programming / August 2007

Limiting upload bandwidth usage