Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: NANOG: users

Question about peering

 

 

NANOG users RSS feed   Index | Next | Previous | View Threaded


me at anuragbhatia

Apr 6, 2012, 12:11 PM

Post #1 of 15 (1227 views)
Permalink
Question about peering

Hello everyone



I am curious to know how small ISPs plan peering with other interested
parties. E.g if ISP A is connected to ISP C via big backbone ISP B, and say
A and C both have open peering policy and assuming the exist in same
exchange or nearby. Now at this point is there is any "minimum bandwidth"
considerations? Say if A and C have 1Gbps + of flowing traffic - very
likely peering would be good idea to save transit costs to B. But if A and
C have very low levels - does it still makes sense? Does peering costs
anything if ISPs are in same exchange? Does at low traffic level it makes
more sense to keep on reaching other ISPs via big transit provider?



Thanks.

--

Anurag Bhatia
anuragbhatia.com
or simply - http://[2001:470:26:78f::5] if you are on IPv6 connected
network!

Twitter: @anurag_bhatia <https://twitter.com/#!/anurag_bhatia>
Linkedin: http://linkedin.anuragbhatia.com


ops.lists at gmail

Apr 6, 2012, 6:45 PM

Post #2 of 15 (1181 views)
Permalink
Re: Question about peering [In reply to]

what does it cost you to peer, versus what does it cost you to not peer?

if you are at the same ix the costs of peering are very low indeed

On Saturday, April 7, 2012, Anurag Bhatia wrote:

> Hello everyone
>
>
>
> I am curious to know how small ISPs plan peering with other interested
> parties. E.g if ISP A is connected to ISP C via big backbone ISP B, and say
> A and C both have open peering policy and assuming the exist in same
> exchange or nearby. Now at this point is there is any "minimum bandwidth"
> considerations? Say if A and C have 1Gbps + of flowing traffic - very
> likely peering would be good idea to save transit costs to B. But if A and
> C have very low levels - does it still makes sense? Does peering costs
> anything if ISPs are in same exchange? Does at low traffic level it makes
> more sense to keep on reaching other ISPs via big transit provider?
>
>
>
> Thanks.
>
> --
>
> Anurag Bhatia
> anuragbhatia.com
> or simply - http://[2001:470:26:78f::5] if you are on IPv6 connected
> network!
>
> Twitter: @anurag_bhatia <https://twitter.com/#!/anurag_bhatia>
> Linkedin: http://linkedin.anuragbhatia.com
>


--
Suresh Ramasubramanian (ops.lists [at] gmail)


jamie at macisa

Apr 7, 2012, 12:32 AM

Post #3 of 15 (1179 views)
Permalink
Re: Question about peering [In reply to]

On 6 Apr 2012, at 20:11, Anurag Bhatia wrote:

> I am curious to know how small ISPs plan peering with other interested
> parties.

<snip>

Hi,

It's not the precise answer you're probably after, but I found the "Internet Peering Playbook" (http://drpeering.net/core/buyTheBook.html) to be full of examples of the sort of question you've asked.

Can't remember where I found out about it (so apologies if this isn't news to you), but it did answer _many_ of the questions I had.

Cheers,
jmi

--
http://jamie.macisa.ac
mailto jamie [at] macisa
mobile +44 7715 707078
gnupg 1024D/A9E61DBE


rs at seastrom

Apr 7, 2012, 3:16 PM

Post #4 of 15 (1179 views)
Permalink
Re: Question about peering [In reply to]

Actually, Suresh, I disagree. It depends on the
facility/country/continent, the cost of joining the local IX fabric at
a reasonable bandwidth, your cost model, and your transit costs. In
short, it's not 1999 anymore, and peering is not automatically the
right answer from a purely fiscal perspective (though it may be from a
technical perspective; see below).

At certain IXes that have a perfect storm of high priced ports and a
good assortment of carriers with sufficiently high quality service and
aggressive pricing, a good negotiator can fairly easily find himself
in a position where the actual cost per megabit of traffic moved on
peered bandwidth exceeds the cost of traffic moved on transit _by an
order of magnitude_. That's without even factoring in the (low)
maintenance cost of having a bunch of BGP sessions around or upgraded
routers or whatever.

Sometimes making the AS path as short as possible makes a lot of sense
(e.g. when trying to get an anycast network to do the right thing),
but assumptions that peering results in lower costs are less true
every day.

-r

Suresh Ramasubramanian <ops.lists [at] gmail> writes:

> what does it cost you to peer, versus what does it cost you to not peer?
>
> if you are at the same ix the costs of peering are very low indeed
>
> On Saturday, April 7, 2012, Anurag Bhatia wrote:
>
>> Hello everyone
>>
>>
>>
>> I am curious to know how small ISPs plan peering with other interested
>> parties. E.g if ISP A is connected to ISP C via big backbone ISP B, and say
>> A and C both have open peering policy and assuming the exist in same
>> exchange or nearby. Now at this point is there is any "minimum bandwidth"
>> considerations? Say if A and C have 1Gbps + of flowing traffic - very
>> likely peering would be good idea to save transit costs to B. But if A and
>> C have very low levels - does it still makes sense? Does peering costs
>> anything if ISPs are in same exchange? Does at low traffic level it makes
>> more sense to keep on reaching other ISPs via big transit provider?
>>
>>
>>
>> Thanks.
>>
>> --
>>
>> Anurag Bhatia
>> anuragbhatia.com
>> or simply - http://[2001:470:26:78f::5] if you are on IPv6 connected
>> network!
>>
>> Twitter: @anurag_bhatia <https://twitter.com/#!/anurag_bhatia>
>> Linkedin: http://linkedin.anuragbhatia.com
>>
>
>
> --
> Suresh Ramasubramanian (ops.lists [at] gmail)


lsc at prgmr

Apr 7, 2012, 3:35 PM

Post #5 of 15 (1177 views)
Permalink
Re: Question about peering [In reply to]

On Sat, Apr 07, 2012 at 06:16:30PM -0400, Robert E. Seastrom wrote:
> Sometimes making the AS path as short as possible makes a lot of sense
> (e.g. when trying to get an anycast network to do the right thing),
> but assumptions that peering results in lower costs are less true
> every day.

I keep reading people say that. But wouldn't the same forces that push
down the per-megabit cost of transit also push down the per-megabit
cost of peering?


rs at seastrom

Apr 7, 2012, 4:25 PM

Post #6 of 15 (1180 views)
Permalink
Re: Question about peering [In reply to]

"Luke S. Crawford" <lsc [at] prgmr> writes:

> On Sat, Apr 07, 2012 at 06:16:30PM -0400, Robert E. Seastrom wrote:
>> Sometimes making the AS path as short as possible makes a lot of sense
>> (e.g. when trying to get an anycast network to do the right thing),
>> but assumptions that peering results in lower costs are less true
>> every day.
>
> I keep reading people say that. But wouldn't the same forces that push
> down the per-megabit cost of transit also push down the per-megabit
> cost of peering?

Generally the costs of transit are pushed down by competition. As a
vendor your costs for bandwidth/transport/port*bw may drop but you are
unlikely to drop your prices to your customers merely because your
costs have gone down unless prompted to by a competitor.

In any given IX, cross-connect fibers and peering switch ports are
often a monopoly. While not unheard-of for there to be two competing
IX switch fabrics available in a single facility, the cross-connects
to those competing exchanges are not free, and I'm not aware of any
sizeabe facilities that are still "run your own XC and don't pay
anyone for it" (of course, as soon as I say that I'll get private
email or an IRC message pointing out the corner case).

Consider the case of a peering n00b network (the target of this
discussion after all) in hypothetical facility that charges
$1000/month for a gigabit ethernet port on the peering fabric. You
turn up a connection to this port and discover that (without buying
people drinks / sushi dinners / etc at a conference) you can bring up
enough peering with other networks to move 150 Mbit/sec on it. That's
pretty optimistic for a small player, but still... now you're paying
$6.66/mbit for that transit. If you can move 150 Mbit/sec to
low-hanging-fruit transit you're probably between 1 and 2gbps total.
How's that compare with what you're paying for transit with that level
of commit?

-r


randy at psg

Apr 7, 2012, 4:48 PM

Post #7 of 15 (1171 views)
Permalink
Re: Question about peering [In reply to]

> wouldn't the same forces that push down the per-megabit cost of
> transit also push down the per-megabit cost of peering?

at some point in the race to the bottom, the cost of a port plus the
opex to maintain a peer becomes a significant factor.

randy


lsc at prgmr

Apr 7, 2012, 5:34 PM

Post #8 of 15 (1173 views)
Permalink
Re: Question about peering [In reply to]

On Sat, Apr 07, 2012 at 07:25:24PM -0400, Robert E. Seastrom wrote:
> Generally the costs of transit are pushed down by competition. As a
> vendor your costs for bandwidth/transport/port*bw may drop but you are
> unlikely to drop your prices to your customers merely because your
> costs have gone down unless prompted to by a competitor.

ah, so it's not the cost of production that is the problem, it is
the 'natural monopoly' state of an IX that is the problem.

It seems like that problem could be overcome by making the
IX a cooperative owned by the members, maybe?

> Consider the case of a peering n00b network (the target of this
> discussion after all) in hypothetical facility that charges
> $1000/month for a gigabit ethernet port on the peering fabric. You

I am in almost that exact position (A peering n00b network) - Of
couse, I'm fairly certain I'm paying sucker prices, but I can get a
gigE to any2 at 55 s market for less than a third the price you quote.

just a data point.


Joel.Snyder at Opus1

Apr 7, 2012, 5:51 PM

Post #9 of 15 (1172 views)
Permalink
Re: Question about peering [In reply to]

>It seems like that problem could be overcome by making the
>IX a cooperative owned by the members, maybe?

Even if an IX is a cooperative, that doesn't say anything about their
costs and the costs of interconnection. Networks and buildings and
cross-connects can get cheap for lots of reasons, but the nature of the
ownership isn't really a factor. Cooperatives can be as poorly run or
have as high costs as any commercial facility.

In fact, you could argue that without some cross-subsidy of co-lo or one
of the providers 'donating' space, a small cooperative is likely to be
more expensive to put together than a large colo facility that has lots
of revenue streams. Or you could argue the opposite. I'm just pointing
out that motivation and ownership don't necessarily dictate final costs.

That being said...

>I am in almost that exact position (A peering n00b network) - Of
>couse, I'm fairly certain I'm paying sucker prices, but I can get a
>gigE to any2 at 55 s market for less than a third the price you quote.

Well, bully for you, but at this very instant I'm looking at a contract
from PCCW which has a component of a cross-connect in Telecity London
(Harbour Exchange) where the cross-connect has been priced out at USD
2400/month (maybe that also includes 1U of space; it's hard to tell).

I do understand that this is NANOG with emphasis on the "NA" part, and
so costs in other geographies may not be all that interesting, but some
facilities do charge an arm and a leg (or maybe PCCW is screwing us over
on the proposal).

jms

--
Joel M Snyder, 1404 East Lind Road, Tucson, AZ, 85719
Senior Partner, Opus One Phone: +1 520 324 0494
jms [at] Opus1 http://www.opus1.com/jms


rs at seastrom

Apr 7, 2012, 6:20 PM

Post #10 of 15 (1172 views)
Permalink
Re: Question about peering [In reply to]

"Luke S. Crawford" <lsc [at] prgmr> writes:

> On Sat, Apr 07, 2012 at 07:25:24PM -0400, Robert E. Seastrom wrote:
>> Generally the costs of transit are pushed down by competition. As a
>> vendor your costs for bandwidth/transport/port*bw may drop but you are
>> unlikely to drop your prices to your customers merely because your
>> costs have gone down unless prompted to by a competitor.
>
> ah, so it's not the cost of production that is the problem, it is
> the 'natural monopoly' state of an IX that is the problem.
>
> It seems like that problem could be overcome by making the
> IX a cooperative owned by the members, maybe?

The whole datacenter?

>> Consider the case of a peering n00b network (the target of this
>> discussion after all) in hypothetical facility that charges
>> $1000/month for a gigabit ethernet port on the peering fabric. You
>
> I am in almost that exact position (A peering n00b network) - Of
> couse, I'm fairly certain I'm paying sucker prices, but I can get a
> gigE to any2 at 55 s market for less than a third the price you quote.
>
> just a data point.

You might want to analyze peering opportunities there:
https://www.peeringdb.com/private/facility_view.php?id=20&peerParticipantsPrivatesPage=1

and get some netflow data out of your own network to see just how much
traffic you're sending there. Fairly easy to do with only 34
participants.

Excel Will Tell You What To Do (tm vgill)

-r


ops.lists at gmail

Apr 7, 2012, 8:42 PM

Post #11 of 15 (1161 views)
Permalink
Re: Question about peering [In reply to]

fair enough. i was thinking smaller and more localized exchanges rather than the big ones

--srs (iPad)

On 08-Apr-2012, at 3:46, "Robert E. Seastrom" <rs [at] seastrom> wrote:

>
> Actually, Suresh, I disagree. It depends on the
> facility/country/continent, the cost of joining the local IX fabric at
> a reasonable bandwidth, your cost model, and your transit costs. In
> short, it's not 1999 anymore, and peering is not automatically the
> right answer from a purely fiscal perspective (though it may be from a
> technical perspective; see below).
>
> At certain IXes that have a perfect storm of high priced ports and a
> good assortment of carriers with sufficiently high quality service and
> aggressive pricing, a good negotiator can fairly easily find himself
> in a position where the actual cost per megabit of traffic moved on
> peered bandwidth exceeds the cost of traffic moved on transit _by an
> order of magnitude_. That's without even factoring in the (low)
> maintenance cost of having a bunch of BGP sessions around or upgraded
> routers or whatever.
>
> Sometimes making the AS path as short as possible makes a lot of sense
> (e.g. when trying to get an anycast network to do the right thing),
> but assumptions that peering results in lower costs are less true
> every day.
>
> -r
>
> Suresh Ramasubramanian <ops.lists [at] gmail> writes:
>
>> what does it cost you to peer, versus what does it cost you to not peer?
>>
>> if you are at the same ix the costs of peering are very low indeed
>>
>> On Saturday, April 7, 2012, Anurag Bhatia wrote:
>>
>>> Hello everyone
>>>
>>>
>>>
>>> I am curious to know how small ISPs plan peering with other interested
>>> parties. E.g if ISP A is connected to ISP C via big backbone ISP B, and say
>>> A and C both have open peering policy and assuming the exist in same
>>> exchange or nearby. Now at this point is there is any "minimum bandwidth"
>>> considerations? Say if A and C have 1Gbps + of flowing traffic - very
>>> likely peering would be good idea to save transit costs to B. But if A and
>>> C have very low levels - does it still makes sense? Does peering costs
>>> anything if ISPs are in same exchange? Does at low traffic level it makes
>>> more sense to keep on reaching other ISPs via big transit provider?
>>>
>>>
>>>
>>> Thanks.
>>>
>>> --
>>>
>>> Anurag Bhatia
>>> anuragbhatia.com
>>> or simply - http://[2001:470:26:78f::5] if you are on IPv6 connected
>>> network!
>>>
>>> Twitter: @anurag_bhatia <https://twitter.com/#!/anurag_bhatia>
>>> Linkedin: http://linkedin.anuragbhatia.com
>>>
>>
>>
>> --
>> Suresh Ramasubramanian (ops.lists [at] gmail)


szlists at szarka

May 9, 2012, 1:50 PM

Post #12 of 15 (1056 views)
Permalink
Re: Question about peering [In reply to]

On 4/6/2012 3:11 PM, Anurag Bhatia wrote:
> I am curious to know how small ISPs plan peering with other interested
> parties. E.g if ISP A is connected to ISP C via big backbone ISP B, and say
> A and C both have open peering policy and assuming the exist in same
> exchange or nearby. Now at this point is there is any "minimum bandwidth"
> considerations? Say if A and C have 1Gbps + of flowing traffic - very
> likely peering would be good idea to save transit costs to B. But if A and
> C have very low levels - does it still makes sense? Does peering costs
> anything if ISPs are in same exchange? Does at low traffic level it makes
> more sense to keep on reaching other ISPs via big transit provider?

One thing to consider is that peering can benefit both networks not just
because of bandwidth savings, but because (given sufficient clue) they
can deliver better performance and reliability to their mutual customers.


keegan.holley at sungard

May 9, 2012, 5:06 PM

Post #13 of 15 (1054 views)
Permalink
Re: Question about peering [In reply to]

Most of the time no. ISP A and ISP C probably don't have alot of traffic
destined for each other's AS's. Without other peers in an IX sort of model
the link would probably be mostly devoid of (useful) traffic. Although, if
ISP A and C were small regional ISP's and they could get free peering from
someone like netflix that may be worth while, but I digress. Another
interesting occurrence would be if ISP A shifted it's metrics to force it's
transit traffic into ISP C's AS offloading the cost of the eventual ISP B
hop to ISP C. (assuming someone announces the full table) I've also seen
ISP A re-announce ISP C's routes to their upstreams with preferred metrics
in order to make the link "one-sided" and begin billing ISP A.



2012/4/6 Anurag Bhatia <me [at] anuragbhatia>

> Hello everyone
>
>
>
> I am curious to know how small ISPs plan peering with other interested
> parties. E.g if ISP A is connected to ISP C via big backbone ISP B, and say
> A and C both have open peering policy and assuming the exist in same
> exchange or nearby. Now at this point is there is any "minimum bandwidth"
> considerations? Say if A and C have 1Gbps + of flowing traffic - very
> likely peering would be good idea to save transit costs to B. But if A and
> C have very low levels - does it still makes sense? Does peering costs
> anything if ISPs are in same exchange? Does at low traffic level it makes
> more sense to keep on reaching other ISPs via big transit provider?
>
>
>
> Thanks.
>
> --
>
> Anurag Bhatia
> anuragbhatia.com
> or simply - http://[2001:470:26:78f::5] if you are on IPv6 connected
> network!
>
> Twitter: @anurag_bhatia <https://twitter.com/#!/anurag_bhatia>
> Linkedin: http://linkedin.anuragbhatia.com
>
>


lists at mtin

May 10, 2012, 7:23 AM

Post #14 of 15 (1046 views)
Permalink
Re: Question about peering [In reply to]

We are cross-connected with several ISPs at a couple of data centers.
Very helpful in one situation as several of us share a soft-switch.

Justin

-----Original Message-----
From: Rob Szarka <szlists [at] szarka>
Date: Wednesday, May 9, 2012 4:50 PM
To: <nanog [at] nanog>
Subject: Re: Question about peering

>On 4/6/2012 3:11 PM, Anurag Bhatia wrote:
>> I am curious to know how small ISPs plan peering with other interested
>> parties. E.g if ISP A is connected to ISP C via big backbone ISP B, and
>>say
>> A and C both have open peering policy and assuming the exist in same
>> exchange or nearby. Now at this point is there is any "minimum
>>bandwidth"
>> considerations? Say if A and C have 1Gbps + of flowing traffic - very
>> likely peering would be good idea to save transit costs to B. But if A
>>and
>> C have very low levels - does it still makes sense? Does peering costs
>> anything if ISPs are in same exchange? Does at low traffic level it
>>makes
>> more sense to keep on reaching other ISPs via big transit provider?
>
>One thing to consider is that peering can benefit both networks not just
>because of bandwidth savings, but because (given sufficient clue) they
>can deliver better performance and reliability to their mutual customers.
>
>


itsmemattchung at gmail

May 10, 2012, 7:39 AM

Post #15 of 15 (1047 views)
Permalink
Re: Question about peering [In reply to]

At the previous regional ISP i worked for we peered with google, facebook,
yahoo, pandora and several other content providers at Any2 exchange
(coresite). We capitalized on that link since a tremendous amount of our
traffic was destined for those networks.

The cost to join that exchange was relatively cheap compared to what we
were paying for transit. You may want to look for a similar exchange at
your pop.

On Thu, May 10, 2012 at 9:23 AM, Justin Wilson <lists [at] mtin> wrote:

> We are cross-connected with several ISPs at a couple of data
> centers.
> Very helpful in one situation as several of us share a soft-switch.
>
> Justin
>
> -----Original Message-----
> From: Rob Szarka <szlists [at] szarka>
> Date: Wednesday, May 9, 2012 4:50 PM
> To: <nanog [at] nanog>
> Subject: Re: Question about peering
>
> >On 4/6/2012 3:11 PM, Anurag Bhatia wrote:
> >> I am curious to know how small ISPs plan peering with other interested
> >> parties. E.g if ISP A is connected to ISP C via big backbone ISP B, and
> >>say
> >> A and C both have open peering policy and assuming the exist in same
> >> exchange or nearby. Now at this point is there is any "minimum
> >>bandwidth"
> >> considerations? Say if A and C have 1Gbps + of flowing traffic - very
> >> likely peering would be good idea to save transit costs to B. But if A
> >>and
> >> C have very low levels - does it still makes sense? Does peering costs
> >> anything if ISPs are in same exchange? Does at low traffic level it
> >>makes
> >> more sense to keep on reaching other ISPs via big transit provider?
> >
> >One thing to consider is that peering can benefit both networks not just
> >because of bandwidth savings, but because (given sufficient clue) they
> >can deliver better performance and reliability to their mutual customers.
> >
> >
>
>
>
>


--
-Matt Chung

NANOG users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.