Mailing List Archive: NANOG: users

P2P traffic optimization Was: Lies, Damned Lies, and Statistics [Was: Re: ATT VP: Internet to hit capacity by 2010]

Index | Next | Previous | View Threaded

laird at pando

Apr 23, 2008, 12:50 PM

Post #1 of 4 (408 views)
Permalink

P2P traffic optimization Was: Lies, Damned Lies, and Statistics [Was: Re: ATT VP: Internet to hit capacity by 2010]

On Apr 23, 2008, at 2:17 PM, Christopher Morrow wrote:
> On Wed, Apr 23, 2008 at 11:39 AM, Alexander Harrowell
> <a.harrowell [at] gmail> wrote:
>>
>>
>>
>> On Wed, Apr 23, 2008 at 3:47 PM, Christopher Morrow
>> <christopher.morrow [at] gmail> wrote:
>>>
>>> It strikes me that often just doing a reverse lookup on the peer
>>> address would be 'good enough' to keep things more 'local' in a
>>> network sense. Something like:
>>>
>>> 1) prefer peers with PTR's like mine (perhaps get address from a
>>> public-ish server - myipaddress.com/ipchicken.com/dshield.org)
>>> 2) prefer peers within my /24->/16 ?
>>>
>>> This does depend on what you define as 'local' as well, 'stay off my
>>> transit links' or 'stay off my last-mile' or 'stay off that godawful
>>> expensive VZ link from CHI to NYC in my backhaul network...
>>
>> Well. here's your problem; depending on the architecture, the IP
>> addressing
>> structure doesn't necessarily map to the network's cost structure.
>> This is
>> why I prefer the P4P/DillTorrent announcement model.
>>
>
> sure 80/20 rule... less complexity in the clients and some benefit(s).
> perhaps short term something like the above with longer term more
> realtime info about locality.

For the applications, it's a lot less work to use a clean network map
from ISP's than it is to in effect derive one from lookups to ASN, /
24, /16, pings, traceroutes, etc. The main reason to spend the effort
to implement those tactics is that it's better than not doing
anything. :-)

Laird Popkin
CTO, Pando Networks
520 Broadway, 10th floor
New York, NY 10012

laird [at] pando
c) 646/465-0570

_______________________________________________
NANOG mailing list
NANOG [at] nanog
http://mailman.nanog.org/mailman/listinfo/nanog

morrowc.lists at gmail

Apr 23, 2008, 2:14 PM

Post #2 of 4 (380 views)
Permalink

Re: P2P traffic optimization Was: Lies, Damned Lies, and Statistics [Was: Re: ATT VP: Internet to hit capacity by 2010] [In reply to]

On Wed, Apr 23, 2008 at 3:50 PM, Laird Popkin <laird [at] pando> wrote:
>
> On Apr 23, 2008, at 2:17 PM, Christopher Morrow wrote:
>
> > On Wed, Apr 23, 2008 at 11:39 AM, Alexander Harrowell
> > <a.harrowell [at] gmail> wrote:
> >
> > >
> > >
> > >
> > > On Wed, Apr 23, 2008 at 3:47 PM, Christopher Morrow
> > > <christopher.morrow [at] gmail> wrote:
> > >
> > > >
> > > > It strikes me that often just doing a reverse lookup on the peer
> > > > address would be 'good enough' to keep things more 'local' in a
> > > > network sense. Something like:
> > > >
> > > > 1) prefer peers with PTR's like mine (perhaps get address from a
> > > > public-ish server - myipaddress.com/ipchicken.com/dshield.org)
> > > > 2) prefer peers within my /24->/16 ?
> > > >
> > > > This does depend on what you define as 'local' as well, 'stay off my
> > > > transit links' or 'stay off my last-mile' or 'stay off that godawful
> > > > expensive VZ link from CHI to NYC in my backhaul network...
> > > >
> > >
> > > Well. here's your problem; depending on the architecture, the IP
> addressing
> > > structure doesn't necessarily map to the network's cost structure. This
> is
> > > why I prefer the P4P/DillTorrent announcement model.
> > >
> > >
> >
> > sure 80/20 rule... less complexity in the clients and some benefit(s).
> > perhaps short term something like the above with longer term more
> > realtime info about locality.
> >
>
> For the applications, it's a lot less work to use a clean network map from
> ISP's than it is to in effect derive one from lookups to ASN, /24, /16,
> pings, traceroutes, etc. The main reason to spend the effort to implement
> those tactics is that it's better than not doing anything. :-)
>

so.. 'not doing anything' may or may not be a good plan.. bittorrent
works fine today(tm). On the other hand, asking network folks to turn
over 'state secrets' (yes some folks, including doug's company)
believe that their network diagrams/designs/paths are in some way
'secret' or a 'competitive advantage', so that will be a blocking
factor. While, doing simple/easy things initially (most bittorrent
things I've seen have <50 peers certainly there are more in some
cases, but average? > or < than 100? so dns lookups or bit-wise
comparisons seem cheap and easy) that get the progress going seems
like a grand plan.

Being blocked for the 100% solution and not making
progress/showing-benefit seems bad :(

-Chris

_______________________________________________
NANOG mailing list
NANOG [at] nanog
http://mailman.nanog.org/mailman/listinfo/nanog

laird at pando

Apr 23, 2008, 3:30 PM

Post #3 of 4 (374 views)
Permalink

Re: P2P traffic optimization Was: Lies, Damned Lies, and Statistics [Was: Re: ATT VP: Internet to hit capacity by 2010] [In reply to]

I would certainly view the two strategies (reverse engineering network information and getting ISP-provided network information) as being complimentary. As you point out, for any ISP that doesn't provide network data, we're better off figuring out what we can to be smarter than 'random'. So while I prefer getting better data from ISP's, that's not holding us back from doing what we can without that data.

ISP's have been very clear that they regard their network maps as being proprietary for many good reasons. The approach that P4P takes is to have an intermediate server (which we call an iTracker) that processes the network maps and provides abstracted guidance (lists of IP prefixes and percentages) to the p2p networks that allows them to figure out which peers are near each other. The iTracker can be run by the ISP or by a trusted third party, as the ISP prefers.

- Laird Popkin, CTO, Pando Networks
mobile: 646/465-0570

----- Original Message -----
From: "Christopher Morrow" <morrowc.lists [at] gmail>
To: "Laird Popkin" <laird [at] pando>
Cc: "Alexander Harrowell" <a.harrowell [at] gmail>, "Doug Pasko" <doug.pasko [at] verizon>, nanog [at] nanog
Sent: Wednesday, April 23, 2008 5:14:12 PM (GMT-0500) America/New_York
Subject: Re: P2P traffic optimization Was: [Nanog] Lies, Damned Lies, and Statistics [Was: Re: ATT VP: Internet to hit capacity by 2010]

On Wed, Apr 23, 2008 at 3:50 PM, Laird Popkin <laird [at] pando> wrote:
>
> On Apr 23, 2008, at 2:17 PM, Christopher Morrow wrote:
>
> > On Wed, Apr 23, 2008 at 11:39 AM, Alexander Harrowell
> > <a.harrowell [at] gmail> wrote:
> >
> > >
> > >
> > >
> > > On Wed, Apr 23, 2008 at 3:47 PM, Christopher Morrow
> > > <christopher.morrow [at] gmail> wrote:
> > >
> > > >
> > > > It strikes me that often just doing a reverse lookup on the peer
> > > > address would be 'good enough' to keep things more 'local' in a
> > > > network sense. Something like:
> > > >
> > > > 1) prefer peers with PTR's like mine (perhaps get address from a
> > > > public-ish server - myipaddress.com/ipchicken.com/dshield.org)
> > > > 2) prefer peers within my /24->/16 ?
> > > >
> > > > This does depend on what you define as 'local' as well, 'stay off my
> > > > transit links' or 'stay off my last-mile' or 'stay off that godawful
> > > > expensive VZ link from CHI to NYC in my backhaul network...
> > > >
> > >
> > > Well. here's your problem; depending on the architecture, the IP
> addressing
> > > structure doesn't necessarily map to the network's cost structure. This
> is
> > > why I prefer the P4P/DillTorrent announcement model.
> > >
> > >
> >
> > sure 80/20 rule... less complexity in the clients and some benefit(s).
> > perhaps short term something like the above with longer term more
> > realtime info about locality.
> >
>
> For the applications, it's a lot less work to use a clean network map from
> ISP's than it is to in effect derive one from lookups to ASN, /24, /16,
> pings, traceroutes, etc. The main reason to spend the effort to implement
> those tactics is that it's better than not doing anything. :-)
>

so.. 'not doing anything' may or may not be a good plan.. bittorrent
works fine today(tm). On the other hand, asking network folks to turn
over 'state secrets' (yes some folks, including doug's company)
believe that their network diagrams/designs/paths are in some way
'secret' or a 'competitive advantage', so that will be a blocking
factor. While, doing simple/easy things initially (most bittorrent
things I've seen have <50 peers certainly there are more in some
cases, but average? > or < than 100? so dns lookups or bit-wise
comparisons seem cheap and easy) that get the progress going seems
like a grand plan.

Being blocked for the 100% solution and not making
progress/showing-benefit seems bad :(

-Chris

_______________________________________________
NANOG mailing list
NANOG [at] nanog
http://mailman.nanog.org/mailman/listinfo/nanog

morrowc.lists at gmail

Apr 23, 2008, 4:47 PM

Post #4 of 4 (383 views)
Permalink

Re: P2P traffic optimization Was: Lies, Damned Lies, and Statistics [Was: Re: ATT VP: Internet to hit capacity by 2010] [In reply to]

On Wed, Apr 23, 2008 at 6:30 PM, Laird Popkin <laird [at] pando> wrote:
> I would certainly view the two strategies (reverse engineering network information and getting ISP-
> provided network information) as being complimentary. As you point out, for any ISP that doesn't
> provide network data, we're better off figuring out what we can to be smarter than 'random'. So while I
> prefer getting better data from ISP's, that's not holding us back from doing what we can without that
> data.

ok, sounds better :) or more reasonable, or not immediately doomed to
blockage :) 'more realistic' even.

> ISP's have been very clear that they regard their network maps as being proprietary for many good
> reasons. The approach that P4P takes is to have an intermediate server (which we call an iTracker)
> that processes the network maps and provides abstracted guidance (lists of IP prefixes and
> percentages) to the p2p networks that allows them to figure out which peers are near each other. The > iTracker can be run by the ISP or by a trusted third party, as the ISP prefers.

What's to keep the itracker from being the new 'napster megaserver'? I
suppose if it just trades map info or lookup (ala dns lookups) and
nothing about torrent/share content things are less sensitive from a
privacy perspective. and a single point of failure of the network
perspective.

Latency requirements seem to be interesting for this as well... at
least dependent upon the model for sharing of the mapping data. I'd
think that a lookup model served the client base better (instead of
downloading many large files of maps in order to determine the best
peers to use). There's also a sensitivity to the part of the network
graph and which perspective to use for the client -> peer locality
mapping.

It's interesting at least :)

Thanks!
-Chris

(also, as an aside, your mail client seems to be making each paragraph
one long unbroken line... which drives at least pine and gmail a bit
bonkers...and makes quoting messages a much more manual process than
it should be.)

_______________________________________________
NANOG mailing list
NANOG [at] nanog
http://mailman.nanog.org/mailman/listinfo/nanog

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads